-- Logs begin at Thu 2022-09-01 14:19:51 UTC, end at Thu 2022-09-01 15:15:59 UTC. -- Sep 01 14:44:34.246479 dv-cinder-855058-2-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Sep 01 14:44:34.255493 dv-cinder-855058-2-iscsi devstack@keystone.service[81328]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Sep 01 14:44:34.257524 dv-cinder-855058-2-iscsi devstack@keystone.service[81328]: *** Starting uWSGI 2.0.18-debian (64bit) on [Thu Sep 1 14:44:34 2022] *** Sep 01 14:44:34.257524 dv-cinder-855058-2-iscsi devstack@keystone.service[81328]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Sep 01 14:44:34.257524 dv-cinder-855058-2-iscsi devstack@keystone.service[81328]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Sep 01 14:44:34.257524 dv-cinder-855058-2-iscsi devstack@keystone.service[81328]: nodename: dv-cinder-855058-2-iscsi Sep 01 14:44:34.257524 dv-cinder-855058-2-iscsi devstack@keystone.service[81328]: machine: x86_64 Sep 01 14:44:34.257524 dv-cinder-855058-2-iscsi devstack@keystone.service[81328]: clock source: unix Sep 01 14:44:34.257524 dv-cinder-855058-2-iscsi devstack@keystone.service[81328]: pcre jit disabled Sep 01 14:44:34.257524 dv-cinder-855058-2-iscsi devstack@keystone.service[81328]: detected number of CPU cores: 4 Sep 01 14:44:34.257524 dv-cinder-855058-2-iscsi devstack@keystone.service[81328]: current working directory: / Sep 01 14:44:34.257524 dv-cinder-855058-2-iscsi devstack@keystone.service[81328]: detected binary path: /usr/bin/uwsgi-core Sep 01 14:44:34.257524 dv-cinder-855058-2-iscsi devstack@keystone.service[81328]: your processes number limit is 47680 Sep 01 14:44:34.258170 dv-cinder-855058-2-iscsi devstack@keystone.service[81328]: your memory page size is 4096 bytes Sep 01 14:44:34.258170 dv-cinder-855058-2-iscsi devstack@keystone.service[81328]: detected max file descriptor number: 2048 Sep 01 14:44:34.258170 dv-cinder-855058-2-iscsi devstack@keystone.service[81328]: lock engine: pthread robust mutexes Sep 01 14:44:34.258170 dv-cinder-855058-2-iscsi devstack@keystone.service[81328]: thunder lock: enabled Sep 01 14:44:34.258170 dv-cinder-855058-2-iscsi devstack@keystone.service[81328]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Sep 01 14:44:34.258170 dv-cinder-855058-2-iscsi devstack@keystone.service[81328]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Sep 01 14:44:34.296860 dv-cinder-855058-2-iscsi devstack@keystone.service[81328]: Python main interpreter initialized at 0x56530b5a69c0 Sep 01 14:44:34.296860 dv-cinder-855058-2-iscsi devstack@keystone.service[81328]: python threads support enabled Sep 01 14:44:34.296860 dv-cinder-855058-2-iscsi devstack@keystone.service[81328]: your server socket listen backlog is limited to 100 connections Sep 01 14:44:34.296860 dv-cinder-855058-2-iscsi devstack@keystone.service[81328]: your mercy for graceful operations on workers is 90 seconds Sep 01 14:44:34.297263 dv-cinder-855058-2-iscsi devstack@keystone.service[81328]: mapped 403077 bytes (393 KB) for 2 cores Sep 01 14:44:34.297346 dv-cinder-855058-2-iscsi devstack@keystone.service[81328]: *** Operational MODE: preforking *** Sep 01 14:44:34.297571 dv-cinder-855058-2-iscsi devstack@keystone.service[81328]: *** uWSGI is running in multiple interpreter mode *** Sep 01 14:44:34.297571 dv-cinder-855058-2-iscsi devstack@keystone.service[81328]: spawned uWSGI master process (pid: 81328) Sep 01 14:44:34.297781 dv-cinder-855058-2-iscsi systemd[1]: Started Devstack devstack@keystone.service. Sep 01 14:44:34.302264 dv-cinder-855058-2-iscsi devstack@keystone.service[81328]: spawned uWSGI worker 1 (pid: 81341, cores: 1) Sep 01 14:44:34.302264 dv-cinder-855058-2-iscsi devstack@keystone.service[81328]: spawned uWSGI worker 2 (pid: 81342, cores: 1) Sep 01 14:44:34.302264 dv-cinder-855058-2-iscsi devstack@keystone.service[81328]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Sep 01 14:44:38.598731 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 01 14:44:38.598731 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: from cryptography.utils import int_from_bytes Sep 01 14:44:38.599526 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 01 14:44:38.599526 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: from cryptography.utils import int_from_bytes Sep 01 14:44:38.599936 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 01 14:44:38.599936 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: from cryptography.utils import int_from_bytes Sep 01 14:44:38.600045 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 01 14:44:38.600045 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: from cryptography.utils import int_from_bytes Sep 01 14:44:38.649568 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 01 14:44:38.649710 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 01 14:44:38.649808 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 01 14:44:38.649896 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 01 14:44:38.649974 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] command line args: [] {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 01 14:44:38.650050 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] command line args: [] {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 01 14:44:38.650206 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 01 14:44:38.650311 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 01 14:44:38.650430 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] ================================================================================ {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 01 14:44:38.650515 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] ================================================================================ {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 01 14:44:38.650671 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] admin_token = **** {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 14:44:38.650757 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] admin_token = **** {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 14:44:38.650921 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] config_dir = [] {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 14:44:38.651006 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] config_dir = [] {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 14:44:38.651104 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 14:44:38.651184 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 14:44:38.651284 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] config_source = [] {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 14:44:38.651356 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] config_source = [] {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 14:44:38.651461 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] debug = True {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 14:44:38.651534 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] debug = True {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 14:44:38.651652 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 14:44:38.652922 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 14:44:38.653026 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 14:44:38.653088 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 14:44:38.653146 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 14:44:38.653209 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 14:44:38.653273 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 14:44:38.653332 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 14:44:38.653399 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 14:44:38.653459 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 14:44:38.653521 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] list_limit = None {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 14:44:38.653579 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] list_limit = None {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 14:44:38.653642 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] log_config_append = None {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 14:44:38.653701 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] log_config_append = None {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 14:44:38.653771 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 14:44:38.653835 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 14:44:38.653898 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] log_dir = None {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 14:44:38.653957 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] log_dir = None {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 14:44:38.654014 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] log_file = None {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 14:44:38.654071 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] log_file = None {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 14:44:38.654138 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 14:44:38.654205 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 14:44:38.654301 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 14:44:38.654364 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 14:44:38.654422 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 14:44:38.654490 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 14:44:38.654559 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 14:44:38.654626 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 14:44:38.654691 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 14:44:38.654757 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 14:44:38.654821 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 14:44:38.654886 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 14:44:38.655171 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 14:44:38.655402 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 14:44:38.655523 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 14:44:38.655644 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 14:44:38.655769 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 14:44:38.655894 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 14:44:38.656021 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 14:44:38.656140 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 14:44:38.656259 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 14:44:38.656377 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 14:44:38.656490 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 14:44:38.656618 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 14:44:38.656731 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 14:44:38.656853 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 14:44:38.656972 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 14:44:38.657091 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 14:44:38.657203 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 14:44:38.657324 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 14:44:38.657446 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.33/identity {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 14:44:38.657566 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.33/identity {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 14:44:38.657692 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] publish_errors = False {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 14:44:38.657850 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] publish_errors = False {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 14:44:38.657971 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 14:44:38.658088 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 14:44:38.658207 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 14:44:38.658417 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 14:44:38.658537 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 14:44:38.658641 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 14:44:38.658641 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 14:44:38.658833 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 14:44:38.658935 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 14:44:38.659032 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] standard_threads = False {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 14:44:38.659126 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 14:44:38.659226 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 14:44:38.659326 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] standard_threads = False {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 14:44:38.659428 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 14:44:38.659523 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 14:44:38.659624 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 14:44:38.659724 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 14:44:38.659820 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] use_journal = False {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 14:44:38.659914 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 14:44:38.660063 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] use_json = False {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 14:44:38.660161 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] use_journal = False {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 14:44:38.660260 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] use_stderr = False {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 14:44:38.660361 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] use_json = False {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 14:44:38.660456 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] use_syslog = False {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 14:44:38.660557 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] use_stderr = False {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 14:44:38.660663 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 14:44:38.660760 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] use_syslog = False {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 14:44:38.660854 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.660955 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 14:44:38.661075 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.661195 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.661195 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.661422 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.661422 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.661634 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.661634 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.661855 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.661979 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.662095 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.662208 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.662332 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.662332 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.662526 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.662526 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.662751 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.662863 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.662975 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.663086 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.663200 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.663323 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.663928 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.664045 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.664157 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.664285 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.664400 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.664528 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.664643 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.664756 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.664868 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.664980 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.665092 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.665212 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.665323 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.665444 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.665559 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.666294 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.666387 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.666461 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.666531 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.666606 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.666678 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.666756 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.666827 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.666902 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.666978 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.667049 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.667187 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.667272 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.667389 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.667464 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.667541 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.667612 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.667687 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.667770 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.667842 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.667953 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.668032 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.668108 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.668210 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.668298 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.668444 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.668521 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.668678 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.668754 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.668927 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.669011 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.669178 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.669254 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.669411 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.669488 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.669658 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.669743 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.669888 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.669965 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.670124 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.670199 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.670379 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.670468 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.670610 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.670688 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.670842 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.670925 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.671072 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.671226 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.671300 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.671369 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.671524 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.671601 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.671768 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.671853 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.671997 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.672074 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.672234 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.672309 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.672459 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.672545 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.672692 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.672769 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.672925 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.673008 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.673169 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.673258 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.673397 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.673504 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.673631 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.673737 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.673848 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.673945 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.674083 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.674188 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.674373 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.674483 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.674570 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.674688 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.674764 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.674878 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.675008 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.675086 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.675241 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.675330 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.675477 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.675558 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.675705 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.675792 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.675962 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.676052 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.676201 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.676279 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.676428 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.676544 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.676662 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.676739 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.676894 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.676977 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.677162 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.677244 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.677400 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.677482 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.677644 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.677727 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.677886 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.678039 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.678119 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.678228 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.678365 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.678445 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.678599 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.678677 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.678847 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.678928 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.679141 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.679860 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.679949 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.680150 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.680270 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.680397 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.680486 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.680650 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.680730 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.680891 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.680968 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.681129 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.681213 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.681372 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.681487 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.681607 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.681692 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.681846 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.681934 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.682082 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.682158 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.682376 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.682469 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.682612 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.682699 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.682856 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.682933 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.683096 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.683173 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.683335 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.683488 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.683563 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.683633 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.683797 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.683875 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.684029 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.684114 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.684259 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.684338 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.684486 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.684563 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.684718 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.684799 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.684956 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.685036 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.685179 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.685256 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.685405 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.685482 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.685647 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.685730 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.685882 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.685960 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.686123 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.686202 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.686382 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.686462 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.686640 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.686718 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.686898 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.686983 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.687134 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] auth.external = None {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.687222 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.687363 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] auth.external = None {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.687441 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.687593 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.687678 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.687883 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.687960 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.688113 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.688193 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] auth.password = None {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.688344 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] auth.password = None {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.688422 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] auth.token = None {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.688576 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] auth.token = None {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.688654 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.688813 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.688891 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.689045 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.689130 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.689285 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.689366 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.689513 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.689592 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.689759 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.689845 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.689994 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.690071 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.690221 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.690349 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] credential.caching = True {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.690477 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] credential.caching = True {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.690557 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.690723 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.690800 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.690970 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.691050 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.691217 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.691295 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.691458 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.691535 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.691694 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.691778 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.691926 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.692002 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.692170 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.692250 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.692413 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.692491 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.692648 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.692725 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.692898 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.692975 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.693184 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.693262 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.693426 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.693509 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.693685 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.693762 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.693926 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.694009 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.694170 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] federation.caching = True {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.694264 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.694438 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.694519 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] federation.caching = True {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.694711 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.694791 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.694965 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.695043 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.695198 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.695284 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.695449 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.695529 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.695680 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.695757 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.695913 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.695998 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.696165 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.696242 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.696400 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.696741 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.696918 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.697008 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.697154 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.697233 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.697394 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] identity.caching = True {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.697472 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.697627 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] identity.caching = True {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.697779 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.697875 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.697954 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.698113 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.698293 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.698386 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.698468 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.698594 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.698673 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.698840 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.698917 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.699069 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.699153 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.699304 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.699382 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.699552 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.699631 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.699783 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.699871 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.700020 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.700098 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.700247 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.700325 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.700477 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.700563 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.700721 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.700799 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.700967 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.701045 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.701213 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.701299 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.701460 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.701538 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.701703 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.701780 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.701965 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.702050 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.702201 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.702296 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.702462 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.702539 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.702691 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.702780 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.702926 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.703005 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.703155 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.703231 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.703389 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.703475 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.703630 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.703708 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.703871 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.703948 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.704105 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.704182 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.704336 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.704414 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.704583 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.704661 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.704821 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.704901 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.705055 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.705133 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.705299 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.705383 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.705532 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.705611 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.705764 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.705841 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.705999 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.706119 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.706230 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.706388 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.706487 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.706566 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.706724 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.706802 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.706964 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.707044 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.707203 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.707282 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.707441 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.707519 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.707688 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.707768 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.707922 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.707999 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.708150 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.708228 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.708380 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.708463 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.708626 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.708705 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.708867 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.708944 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.709101 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.709186 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.709337 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.709422 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.709568 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] ldap.user = None {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.709646 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.709796 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] ldap.user = None {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.709874 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.710035 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.710113 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.710327 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.710409 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.710562 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.710642 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.710807 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.710886 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.711053 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.711132 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.711289 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.711369 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.711520 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.711598 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.711748 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.711826 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.711982 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.712061 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.712211 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.712288 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.712442 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.712526 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.712668 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.712747 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.712911 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.712990 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.713155 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.713234 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.713398 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.713478 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.713642 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.713725 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.713873 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.713950 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.714101 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.714179 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.714367 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.714457 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.714604 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.714684 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.714838 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.714917 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.715072 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.715151 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.715307 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.715385 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.715551 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.715631 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.715791 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.715868 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.716023 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.716101 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.716271 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.716355 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.716499 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.716576 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.716742 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.716821 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.716977 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.717062 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.717211 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.717290 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.717445 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.717572 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.717798 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.717878 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.718042 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.718121 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.718293 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.718375 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.718534 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.718614 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] resource.caching = True {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.718776 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] resource.caching = True {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.718854 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.719025 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.719102 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.719264 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.719343 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.719499 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.719576 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.719747 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.719833 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.719989 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.720066 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.720221 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.720298 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.720466 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.720618 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.720694 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.720765 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.720928 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.721007 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] role.caching = True {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.721163 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] role.caching = True {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.721248 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] role.driver = sql {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.721400 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.721477 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] role.driver = sql {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.721627 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.721704 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.721875 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.721961 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.722121 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.722205 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.722393 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.722474 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.722638 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.722728 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.722883 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.722963 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.723128 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.723206 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.723370 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.723456 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.723605 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.723683 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.723835 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.723912 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.724079 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.724165 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.724323 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.724402 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.724567 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.724646 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.724811 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.724969 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.725205 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.725283 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.725440 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.725517 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.725687 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.725773 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.726453 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.726540 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.726712 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.726790 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.726950 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.727029 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.727184 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.727262 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.727443 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.727551 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.727691 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.727767 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.727926 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.728004 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.728159 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.728236 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.728386 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.728464 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.728615 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.728695 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.728865 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.728943 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.729117 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.729196 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.729353 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.729430 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.729587 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.729664 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.729819 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.729897 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.730048 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.730125 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] token.caching = True {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.730305 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] token.caching = True {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.730388 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.730543 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.730623 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.730783 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.730861 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.731016 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.731094 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.731270 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.731350 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.731509 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.731586 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.731741 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.731818 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.731982 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.732061 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.732224 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.732302 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.732461 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.732538 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.732693 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.732771 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.732927 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.733004 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.733157 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.733234 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.733401 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.733481 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.733666 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.733758 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.733893 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.733980 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.734135 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.734215 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.734435 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.734527 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] database.connection = **** {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.734681 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.734759 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] database.connection = **** {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.734924 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.735002 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.735164 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.735250 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.735397 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.735475 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.735631 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.735708 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.735863 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.735948 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.736099 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.736338 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.736416 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.736586 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.736664 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.736820 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.736898 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.737063 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.737140 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.737295 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.737372 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.737536 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.737616 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.737770 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.737848 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.738015 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.738092 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.738277 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.738360 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.738513 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.738591 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.738740 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.738817 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.738901 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 01 14:44:38.738975 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.739197 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 14:44:38.739365 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81341) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 01 14:44:38.768740 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81342) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 14:44:38.768972 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81342) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 14:44:38.769203 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81342) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 14:44:38.769365 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81341) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 14:44:38.769594 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81341) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 14:44:38.769819 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81341) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 14:44:38.777651 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81342) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 14:44:38.778412 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81341) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 14:44:38.807021 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81342) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 14:44:38.807315 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81342) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 14:44:38.809834 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81341) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 14:44:38.809834 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81341) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 14:44:38.861924 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81342) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 14:44:38.862128 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81341) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 14:44:38.910515 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81341) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 14:44:38.910750 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81341) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 14:44:38.910888 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81342) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 14:44:38.911118 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81342) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 14:44:39.069291 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81342) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 14:44:39.069558 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81342) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 14:44:39.069691 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81341) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 14:44:39.069823 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81342) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 14:44:39.069967 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81341) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 14:44:39.070096 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81342) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 14:44:39.070221 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81341) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 14:44:39.070373 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81342) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 14:44:39.070502 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81341) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 14:44:39.070655 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81342) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 14:44:39.070784 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81341) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 14:44:39.070784 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81341) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 14:44:39.094108 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81342) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 14:44:39.094369 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81342) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 14:44:39.094501 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81341) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 14:44:39.094641 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81342) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 14:44:39.094766 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81341) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 14:44:39.094896 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81342) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 14:44:39.095025 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81341) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 14:44:39.095154 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81342) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 14:44:39.095302 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81341) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 14:44:39.095432 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81342) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 14:44:39.095560 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81341) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 14:44:39.095689 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81342) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 14:44:39.095817 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81341) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 14:44:39.095817 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81341) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 14:44:39.107066 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81342) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 14:44:39.107312 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81341) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 14:44:39.119945 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81342) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 14:44:39.120060 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81341) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 14:44:39.120152 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81342) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 14:44:39.120281 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81341) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 14:44:39.120366 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81342) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 14:44:39.120507 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81341) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 14:44:39.120589 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81342) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 14:44:39.120728 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81341) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 14:44:39.120820 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81342) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 14:44:39.120947 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81341) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 14:44:39.121030 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81342) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 14:44:39.121163 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81341) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 14:44:39.121253 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81342) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 14:44:39.121379 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81341) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 14:44:39.138215 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 14:44:39.138456 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 01 14:44:39.138679 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81341) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 14:44:39.138803 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 14:44:39.138889 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81341) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 01 14:44:39.139007 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 01 14:44:39.139225 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81341) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 14:44:39.139352 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 14:44:39.139471 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81341) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 01 14:44:39.139589 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 14:44:39.139716 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81341) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 14:44:39.139840 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 14:44:39.139947 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81341) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 14:44:39.140031 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 14:44:39.140209 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81341) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 14:44:39.140471 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81341) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 14:44:39.150159 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 14:44:39.150480 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 14:44:39.150563 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81341) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 14:44:39.150754 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 14:44:39.150838 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81341) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 14:44:39.151046 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 14:44:39.151129 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81341) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 14:44:39.151341 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81341) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 14:44:39.157996 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81342) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 14:44:39.158306 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81341) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 14:44:39.160935 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81342) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 14:44:39.161193 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81341) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 14:44:39.161276 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 14:44:39.161480 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81341) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 14:44:39.161561 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 14:44:39.161769 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81341) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 14:44:39.161860 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 14:44:39.162052 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81341) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 14:44:39.162134 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 14:44:39.162342 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 14:44:39.162425 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81341) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 14:44:39.162618 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81341) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 14:44:39.162700 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 14:44:39.162889 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81341) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 14:44:39.162971 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 14:44:39.163165 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 14:44:39.163253 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81341) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 14:44:39.163497 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 14:44:39.164076 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81341) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 14:44:39.164275 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 14:44:39.164355 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81341) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 14:44:39.164587 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81341) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 14:44:39.186955 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 14:44:39.187336 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81341) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 14:44:39.188428 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81342) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 14:44:39.188717 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 14:44:39.188798 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81341) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 14:44:39.189069 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81341) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 14:44:39.193270 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81342) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 14:44:39.193563 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 14:44:39.193645 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81341) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 14:44:39.193853 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 14:44:39.193947 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81341) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 14:44:39.194153 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81341) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 14:44:39.200653 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81342) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 14:44:39.200924 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81341) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 14:44:39.201008 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 14:44:39.201215 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81341) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 14:44:39.204901 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81342) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 14:44:39.205198 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 14:44:39.205279 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81341) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 14:44:39.205487 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 14:44:39.205569 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81341) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 14:44:39.205781 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81341) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 14:44:39.205862 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 14:44:39.206079 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81341) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 14:44:39.206171 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 14:44:39.206380 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81341) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 14:44:39.208053 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 14:44:39.208303 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81341) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 14:44:39.208385 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 14:44:39.208603 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81341) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 14:44:39.208685 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 14:44:39.208895 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81341) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 14:44:39.224829 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 14:44:39.225088 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81341) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 14:44:39.225173 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 14:44:39.225370 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81341) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 14:44:39.229594 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81342) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 14:44:39.229819 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81341) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 14:44:39.232736 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 14:44:39.232962 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81341) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 14:44:39.233045 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 14:44:39.233261 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81341) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 14:44:39.237373 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81342) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 14:44:39.237604 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81341) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 14:44:39.241382 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81341) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 14:44:39.241568 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81342) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 14:44:39.244599 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81341) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 14:44:39.244751 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 14:44:39.244903 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81341) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 14:44:39.245061 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 14:44:39.245189 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81341) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 14:44:39.245355 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 14:44:39.245473 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81341) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 14:44:39.245649 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 14:44:39.245754 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81341) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 14:44:39.245935 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 14:44:39.246142 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81341) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 14:44:39.246352 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 14:44:39.261985 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81341) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 14:44:39.262287 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81342) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 14:44:39.262371 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81341) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 14:44:39.262566 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81341) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 14:44:39.262646 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 14:44:39.262832 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81341) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 14:44:39.262915 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 14:44:39.263102 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 14:44:39.269544 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81341) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 14:44:39.269857 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 14:44:39.271244 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81341) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 14:44:39.271566 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 14:44:39.273316 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81341) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 14:44:39.273601 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81341) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 14:44:39.273683 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 14:44:39.273921 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 14:44:39.276273 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81341) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 14:44:39.276563 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81341) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 14:44:39.276645 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81342) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 14:44:39.276857 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81341) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 14:44:39.276939 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 14:44:39.277152 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 14:44:39.277235 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81341) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 14:44:39.277439 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81341) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 14:44:39.277529 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 14:44:39.277731 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 14:44:39.289558 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81341) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 14:44:39.289701 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81342) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 14:44:39.289843 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81341) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 14:44:39.289992 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 14:44:39.290126 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81341) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 14:44:39.290302 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 14:44:39.290437 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81341) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 14:44:39.290591 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 14:44:39.290722 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81341) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 14:44:39.290878 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 14:44:39.291001 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81341) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 14:44:39.291159 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 14:44:39.291300 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81341) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 14:44:39.291457 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 14:44:39.306708 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81341) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 14:44:39.306958 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81342) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 14:44:39.309737 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81341) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 14:44:39.309999 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81342) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 14:44:39.312651 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81341) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 14:44:39.312927 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 14:44:39.314084 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81341) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 14:44:39.314394 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 14:44:39.315595 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81341) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 14:44:39.315881 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81341) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 14:44:39.315963 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81342) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 14:44:39.316168 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 14:44:39.316299 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81341) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 14:44:39.316465 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 14:44:39.323557 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81341) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 14:44:39.323872 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 14:44:39.325027 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81341) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 14:44:39.325360 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81342) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 14:44:39.327912 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81341) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 14:44:39.328213 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81341) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 14:44:39.328295 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 14:44:39.328501 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81341) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 14:44:39.328595 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 14:44:39.328780 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81341) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 14:44:39.328862 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 14:44:39.329101 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 14:44:39.337239 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81341) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 14:44:39.337531 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81341) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 14:44:39.337765 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81342) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 14:44:39.337848 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81341) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 14:44:39.338063 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 14:44:39.338374 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 14:44:39.344722 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81341) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 14:44:39.345012 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81341) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 14:44:39.345297 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81341) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 14:44:39.345379 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81342) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 14:44:39.345577 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81341) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 14:44:39.345669 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 14:44:39.345853 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81341) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 14:44:39.345944 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 14:44:39.346132 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 14:44:39.346213 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81341) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 14:44:39.346434 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81341) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 14:44:39.346520 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 14:44:39.346710 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 14:44:39.346801 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81341) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 14:44:39.346988 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 14:44:39.347071 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81341) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 14:44:39.347267 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 14:44:39.347350 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81341) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 14:44:39.347537 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81341) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 14:44:39.347618 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 14:44:39.347800 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81341) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 14:44:39.347925 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 14:44:39.348066 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81341) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 14:44:39.348148 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 14:44:39.348337 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 14:44:39.348416 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81341) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 14:44:39.348603 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 14:44:39.348882 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 14:44:39.382138 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x56530b5a69c0 pid: 81341 (default app) Sep 01 14:44:39.383435 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x56530b5a69c0 pid: 81342 (default app) Sep 01 14:44:39.386096 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c71f56d-ad06-4462-a561-e820a6fb78e6 None None] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:44:39.386418 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c71f56d-ad06-4462-a561-e820a6fb78e6 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:44:39.386695 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c71f56d-ad06-4462-a561-e820a6fb78e6 None None] PATH_INFO: `/v3` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:44:39.390840 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1/1] 10.222.0.33 () {50 vars in 791 bytes} [Thu Sep 1 14:44:39 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:44:45.904947 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b90cd88-5593-4801-9d94-ec061da7afdb None None] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:44:45.905475 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b90cd88-5593-4801-9d94-ec061da7afdb None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:44:45.905475 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b90cd88-5593-4801-9d94-ec061da7afdb None None] PATH_INFO: `/` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:44:45.909852 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1/2] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 1 14:44:45 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 14:44:45.915025 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e57a8b-db7c-4f8d-b266-a8295d37e62e None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:44:45.915503 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e57a8b-db7c-4f8d-b266-a8295d37e62e None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:44:45.915503 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e57a8b-db7c-4f8d-b266-a8295d37e62e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:44:45.983854 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: INFO dbcounter [None req-21e57a8b-db7c-4f8d-b266-a8295d37e62e None None] Registered counter for database keystone Sep 01 14:44:46.002901 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG oslo_db.sqlalchemy.engines [None req-21e57a8b-db7c-4f8d-b266-a8295d37e62e None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81341) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 01 14:44:46.010167 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writer thread running {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 01 14:44:46.187064 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG passlib.handlers.bcrypt [None req-21e57a8b-db7c-4f8d-b266-a8295d37e62e None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81341) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 01 14:44:46.187376 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG passlib.handlers.bcrypt [None req-21e57a8b-db7c-4f8d-b266-a8295d37e62e None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81341) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 01 14:44:46.245203 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-21e57a8b-db7c-4f8d-b266-a8295d37e62e None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:44:46.246817 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-21e57a8b-db7c-4f8d-b266-a8295d37e62e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:44:46.371961 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 2/3] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 1 14:44:45 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 458 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Sep 01 14:44:46.861680 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b90cd88-5593-4801-9d94-ec061da7afdb None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:44:46.929544 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: INFO dbcounter [None req-6b90cd88-5593-4801-9d94-ec061da7afdb None None] Registered counter for database keystone Sep 01 14:44:46.950810 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG oslo_db.sqlalchemy.engines [None req-6b90cd88-5593-4801-9d94-ec061da7afdb None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81342) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 01 14:44:46.957029 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writer thread running {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 01 14:44:47.077192 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b90cd88-5593-4801-9d94-ec061da7afdb None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:44:47.081397 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7989146-b68f-4c6e-968f-445c71a1f63c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:44:47.082009 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7989146-b68f-4c6e-968f-445c71a1f63c admin admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:44:47.082312 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7989146-b68f-4c6e-968f-445c71a1f63c admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:44:47.082583 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7989146-b68f-4c6e-968f-445c71a1f63c admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:44:47.091047 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7989146-b68f-4c6e-968f-445c71a1f63c admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:44:47.142248 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: WARNING keystone.common.rbac_enforcer.enforcer [None req-b7989146-b68f-4c6e-968f-445c71a1f63c admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 01 14:44:47.142606 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7989146-b68f-4c6e-968f-445c71a1f63c admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:44:47.152396 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 2/4] 10.222.0.33 () {58 vars in 1324 bytes} [Thu Sep 1 14:44:46 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 01 14:44:47.158448 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e57a8b-db7c-4f8d-b266-a8295d37e62e None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:44:47.172768 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e57a8b-db7c-4f8d-b266-a8295d37e62e None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:44:47.177124 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7e8295-9513-4373-b897-2490d09840d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:44:47.178204 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7e8295-9513-4373-b897-2490d09840d1 admin admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:44:47.178517 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7e8295-9513-4373-b897-2490d09840d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:44:47.178791 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7e8295-9513-4373-b897-2490d09840d1 admin admin] PATH_INFO: `/v3/projects/5cec9ba4a51d40d1be462a47013baf0a` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:44:47.180555 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7e8295-9513-4373-b897-2490d09840d1 admin admin] RBAC: Authorizing `identity:get_project(project_id=5cec9ba4a51d40d1be462a47013baf0a)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:44:47.233867 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: WARNING keystone.common.rbac_enforcer.enforcer [None req-0f7e8295-9513-4373-b897-2490d09840d1 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 01 14:44:47.234211 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7e8295-9513-4373-b897-2490d09840d1 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:44:47.238801 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 3/5] 10.222.0.33 () {56 vars in 1207 bytes} [Thu Sep 1 14:44:47 2022] GET /identity/v3/projects/5cec9ba4a51d40d1be462a47013baf0a => generated 344 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:44:49.111778 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7ce7cd-6dc3-4ae8-8225-93c03167b1d1 None None] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:44:49.112331 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7ce7cd-6dc3-4ae8-8225-93c03167b1d1 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:44:49.112331 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7ce7cd-6dc3-4ae8-8225-93c03167b1d1 None None] PATH_INFO: `/` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:44:49.116447 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 3/6] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 1 14:44:49 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:44:49.121733 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca0efcd-4d59-4795-8d61-355db290b0c3 None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:44:49.122124 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca0efcd-4d59-4795-8d61-355db290b0c3 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:44:49.122314 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca0efcd-4d59-4795-8d61-355db290b0c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:44:49.142434 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-7ca0efcd-4d59-4795-8d61-355db290b0c3 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:44:49.143960 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-7ca0efcd-4d59-4795-8d61-355db290b0c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:44:49.152384 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 4/7] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 1 14:44:49 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Sep 01 14:44:49.418359 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7ce7cd-6dc3-4ae8-8225-93c03167b1d1 None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:44:49.430756 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7ce7cd-6dc3-4ae8-8225-93c03167b1d1 None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:44:49.434915 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfa6a54-42e6-44eb-a327-daf2b3c9fbcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:44:49.435505 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfa6a54-42e6-44eb-a327-daf2b3c9fbcd admin admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:44:49.435782 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfa6a54-42e6-44eb-a327-daf2b3c9fbcd admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:44:49.436053 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfa6a54-42e6-44eb-a327-daf2b3c9fbcd admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:44:49.443465 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfa6a54-42e6-44eb-a327-daf2b3c9fbcd admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:44:49.445258 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfa6a54-42e6-44eb-a327-daf2b3c9fbcd admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:44:49.453980 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 4/8] 10.222.0.33 () {58 vars in 1324 bytes} [Thu Sep 1 14:44:49 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 01 14:44:49.459122 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca0efcd-4d59-4795-8d61-355db290b0c3 None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:44:49.468647 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca0efcd-4d59-4795-8d61-355db290b0c3 None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:44:49.472958 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4daf98-e100-4287-a11c-2921270c7867 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:44:49.473581 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4daf98-e100-4287-a11c-2921270c7867 admin admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:44:49.473862 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4daf98-e100-4287-a11c-2921270c7867 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:44:49.474135 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4daf98-e100-4287-a11c-2921270c7867 admin admin] PATH_INFO: `/v3/users/abbabc07754b486599ec507190d24c97` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:44:49.475551 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4daf98-e100-4287-a11c-2921270c7867 admin admin] RBAC: Authorizing `identity:get_user(user_id=abbabc07754b486599ec507190d24c97)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:44:49.477947 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4daf98-e100-4287-a11c-2921270c7867 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:44:49.479509 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 5/9] 10.222.0.33 () {56 vars in 1198 bytes} [Thu Sep 1 14:44:49 2022] GET /identity/v3/users/abbabc07754b486599ec507190d24c97 => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 14:44:51.444255 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3725abc5-e0bf-4045-b512-48ec6025d1fb None None] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:44:51.445349 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3725abc5-e0bf-4045-b512-48ec6025d1fb None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:44:51.445505 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3725abc5-e0bf-4045-b512-48ec6025d1fb None None] PATH_INFO: `/` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:44:51.446262 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fca0f51-bd3d-4cb1-9003-7a598259785d None None] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:44:51.446891 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fca0f51-bd3d-4cb1-9003-7a598259785d None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:44:51.447311 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fca0f51-bd3d-4cb1-9003-7a598259785d None None] PATH_INFO: `/` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:44:51.447417 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 5/10] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 1 14:44:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:44:51.448461 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 6/11] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 1 14:44:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:44:51.451878 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb91c96-4752-4d23-a998-710c8e081773 None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:44:51.452236 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb91c96-4752-4d23-a998-710c8e081773 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:44:51.452506 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb91c96-4752-4d23-a998-710c8e081773 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:44:51.453560 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5583fc69-efd2-4b95-869b-41c50f2d6ec2 None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:44:51.453847 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5583fc69-efd2-4b95-869b-41c50f2d6ec2 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:44:51.454122 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5583fc69-efd2-4b95-869b-41c50f2d6ec2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:44:51.472332 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-5583fc69-efd2-4b95-869b-41c50f2d6ec2 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:44:51.473515 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-5583fc69-efd2-4b95-869b-41c50f2d6ec2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:44:51.493896 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG passlib.handlers.bcrypt [None req-aeb91c96-4752-4d23-a998-710c8e081773 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81342) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 01 14:44:51.494317 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG passlib.handlers.bcrypt [None req-aeb91c96-4752-4d23-a998-710c8e081773 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81342) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 01 14:44:51.543544 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-aeb91c96-4752-4d23-a998-710c8e081773 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:44:51.545180 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-aeb91c96-4752-4d23-a998-710c8e081773 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:44:51.551607 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 7/12] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 1 14:44:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 14:44:51.644586 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 6/13] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 1 14:44:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 14:44:51.815050 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5583fc69-efd2-4b95-869b-41c50f2d6ec2 None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:44:51.838913 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:44:51.914295 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb91c96-4752-4d23-a998-710c8e081773 None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:44:51.917722 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5583fc69-efd2-4b95-869b-41c50f2d6ec2 None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:44:51.954557 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03faf4b7-2957-464c-b8ce-cdf94ca8d9ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:44:51.955167 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03faf4b7-2957-464c-b8ce-cdf94ca8d9ae None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:44:51.955449 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03faf4b7-2957-464c-b8ce-cdf94ca8d9ae None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:44:51.955720 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03faf4b7-2957-464c-b8ce-cdf94ca8d9ae None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:44:52.014536 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb91c96-4752-4d23-a998-710c8e081773 None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:44:52.016820 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:44:52.021098 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03faf4b7-2957-464c-b8ce-cdf94ca8d9ae None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:44:52.022657 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03faf4b7-2957-464c-b8ce-cdf94ca8d9ae None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:44:52.045236 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:44:52.052338 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a0313e-8f37-4a08-8b1f-2a3d5d0a5ea6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:44:52.053429 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a0313e-8f37-4a08-8b1f-2a3d5d0a5ea6 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:44:52.054212 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a0313e-8f37-4a08-8b1f-2a3d5d0a5ea6 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:44:52.054518 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a0313e-8f37-4a08-8b1f-2a3d5d0a5ea6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:44:52.066005 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a0313e-8f37-4a08-8b1f-2a3d5d0a5ea6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 14:44:52.075255 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application [None req-97a0313e-8f37-4a08-8b1f-2a3d5d0a5ea6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 14:44:52.075255 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:44:52.075255 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:44:52.075255 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:44:52.075255 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:44:52.075255 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:44:52.075255 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:44:52.075255 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:44:52.075255 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:44:52.075255 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:44:52.075255 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:44:52.075255 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:44:52.075255 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 14:44:52.075255 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 14:44:52.075255 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 14:44:52.075255 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 14:44:52.076438 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 14:44:52.076438 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 14:44:52.076438 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 14:44:52.076438 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 14:44:52.076438 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:44:52.076438 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:44:52.076438 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:44:52.076438 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:44:52.076438 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:44:52.076438 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:44:52.076438 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:44:52.076438 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application with Lock( Sep 01 14:44:52.076438 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:44:52.076438 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:44:52.076438 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:44:52.076438 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:44:52.077628 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:44:52.077628 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:44:52.077628 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:44:52.077628 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:44:52.077628 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 14:44:52.077628 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 14:44:52.077628 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 14:44:52.077628 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 14:44:52.077628 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 14:44:52.077628 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 14:44:52.077628 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 14:44:52.077628 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Sep 01 14:44:52.077628 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 7/14] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Sep 1 14:44:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 14:44:52.082041 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a0313e-8f37-4a08-8b1f-2a3d5d0a5ea6 None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:44:52.124805 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 8/15] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Sep 1 14:44:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 14:44:52.130406 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03faf4b7-2957-464c-b8ce-cdf94ca8d9ae None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:44:52.179498 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a0313e-8f37-4a08-8b1f-2a3d5d0a5ea6 None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:44:52.211929 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b185e0fa-44b8-407e-99f5-f7ae5fff8972 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:44:52.212761 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b185e0fa-44b8-407e-99f5-f7ae5fff8972 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:44:52.213045 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b185e0fa-44b8-407e-99f5-f7ae5fff8972 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:44:52.213316 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b185e0fa-44b8-407e-99f5-f7ae5fff8972 None admin] PATH_INFO: `/v3/roles` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:44:52.213979 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b185e0fa-44b8-407e-99f5-f7ae5fff8972 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:44:52.214305 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b185e0fa-44b8-407e-99f5-f7ae5fff8972 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:44:52.215749 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b185e0fa-44b8-407e-99f5-f7ae5fff8972 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:44:52.225528 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03faf4b7-2957-464c-b8ce-cdf94ca8d9ae None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:44:52.227610 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 8/16] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Sep 1 14:44:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 14:44:52.233128 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b185e0fa-44b8-407e-99f5-f7ae5fff8972 None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:44:52.250908 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:44:52.260754 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:44:52.263472 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1a7cc2-9579-4220-a2a7-d4ad09adb673 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:44:52.264136 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1a7cc2-9579-4220-a2a7-d4ad09adb673 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:44:52.264417 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1a7cc2-9579-4220-a2a7-d4ad09adb673 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:44:52.264689 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1a7cc2-9579-4220-a2a7-d4ad09adb673 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:44:52.274434 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1a7cc2-9579-4220-a2a7-d4ad09adb673 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:44:52.275937 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1a7cc2-9579-4220-a2a7-d4ad09adb673 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:44:52.294025 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application [None req-bb1a7cc2-9579-4220-a2a7-d4ad09adb673 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 14:44:52.294025 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:44:52.294025 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:44:52.294025 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:44:52.294025 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:44:52.294025 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:44:52.294025 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:44:52.294025 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:44:52.294025 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:44:52.294025 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:44:52.294025 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:44:52.294025 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:44:52.294025 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 14:44:52.294025 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 14:44:52.294025 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 14:44:52.294025 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 14:44:52.295082 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:44:52.295082 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:44:52.295082 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:44:52.295082 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:44:52.295082 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:44:52.295082 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:44:52.295082 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:44:52.295082 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application with Lock( Sep 01 14:44:52.295082 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:44:52.295082 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:44:52.295082 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:44:52.295082 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:44:52.295082 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:44:52.295082 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:44:52.295082 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:44:52.295082 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:44:52.295082 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 14:44:52.296052 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 14:44:52.296052 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 14:44:52.296052 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Sep 01 14:44:52.296052 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 9/17] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 1 14:44:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 14:44:52.300780 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1a7cc2-9579-4220-a2a7-d4ad09adb673 None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:44:52.334522 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b185e0fa-44b8-407e-99f5-f7ae5fff8972 None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:44:52.368245 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3004c3ac-d599-405f-b3b6-15964564f61f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:44:52.368871 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3004c3ac-d599-405f-b3b6-15964564f61f None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:44:52.369155 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3004c3ac-d599-405f-b3b6-15964564f61f None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:44:52.369422 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3004c3ac-d599-405f-b3b6-15964564f61f None admin] PATH_INFO: `/v3/users/abbabc07754b486599ec507190d24c97` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:44:52.370918 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3004c3ac-d599-405f-b3b6-15964564f61f None admin] RBAC: Authorizing `identity:get_user(user_id=abbabc07754b486599ec507190d24c97)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:44:52.372374 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3004c3ac-d599-405f-b3b6-15964564f61f None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:44:52.373925 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 9/18] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Sep 1 14:44:52 2022] GET /identity/v3/users/abbabc07754b486599ec507190d24c97 => generated 247 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 14:44:52.379347 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3004c3ac-d599-405f-b3b6-15964564f61f None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:44:52.398954 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1a7cc2-9579-4220-a2a7-d4ad09adb673 None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:44:52.431954 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2bf998-161b-4d90-a58b-df2df75f80fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:44:52.432545 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2bf998-161b-4d90-a58b-df2df75f80fc None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:44:52.432823 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2bf998-161b-4d90-a58b-df2df75f80fc None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:44:52.433105 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2bf998-161b-4d90-a58b-df2df75f80fc None admin] PATH_INFO: `/v3/domains` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:44:52.433739 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2bf998-161b-4d90-a58b-df2df75f80fc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81341) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:44:52.434036 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2bf998-161b-4d90-a58b-df2df75f80fc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:44:52.435458 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2bf998-161b-4d90-a58b-df2df75f80fc None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:44:52.453158 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 10/19] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Sep 1 14:44:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 14:44:52.474900 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:44:52.483318 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3004c3ac-d599-405f-b3b6-15964564f61f None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:44:52.519395 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effc8ed3-5846-45ca-b74e-556a68db46f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:44:52.520044 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effc8ed3-5846-45ca-b74e-556a68db46f4 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:44:52.520324 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effc8ed3-5846-45ca-b74e-556a68db46f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:44:52.520596 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effc8ed3-5846-45ca-b74e-556a68db46f4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:44:52.521766 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effc8ed3-5846-45ca-b74e-556a68db46f4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:44:52.523807 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effc8ed3-5846-45ca-b74e-556a68db46f4 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:44:52.525579 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 10/20] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 1 14:44:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 14:44:52.531587 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2bf998-161b-4d90-a58b-df2df75f80fc None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:44:52.566820 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:44:52.634954 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2bf998-161b-4d90-a58b-df2df75f80fc None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:44:52.675596 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ad5650-a953-4d66-8ee8-111a7ee0907d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:44:52.676216 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ad5650-a953-4d66-8ee8-111a7ee0907d None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:44:52.676498 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ad5650-a953-4d66-8ee8-111a7ee0907d None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:44:52.676773 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ad5650-a953-4d66-8ee8-111a7ee0907d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:44:52.677508 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ad5650-a953-4d66-8ee8-111a7ee0907d None admin] RBAC: Adding query filter params (role.id=2a2adb769e55476b82e5014897e0a9c0, scope.domain.id=default, user.id=abbabc07754b486599ec507190d24c97) {{(pid=81341) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:44:52.677816 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ad5650-a953-4d66-8ee8-111a7ee0907d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:44:52.679443 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ad5650-a953-4d66-8ee8-111a7ee0907d None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:44:52.686792 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 11/21] 10.222.0.33 () {56 vars in 1322 bytes} [Thu Sep 1 14:44:52 2022] GET /identity/v3/role_assignments?user.id=abbabc07754b486599ec507190d24c97&scope.domain.id=default&role.id=2a2adb769e55476b82e5014897e0a9c0 => generated 233 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:44:54.709658 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3aaccb-24d1-458e-a65f-67b321a8e4e3 None None] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:44:54.711203 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3aaccb-24d1-458e-a65f-67b321a8e4e3 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:44:54.711678 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3aaccb-24d1-458e-a65f-67b321a8e4e3 None None] PATH_INFO: `/` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:44:54.713467 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 11/22] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 1 14:44:54 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:44:54.719149 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d623d87c-db90-40ab-9064-5551c89dc046 None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:44:54.719989 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d623d87c-db90-40ab-9064-5551c89dc046 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:44:54.720475 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d623d87c-db90-40ab-9064-5551c89dc046 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:44:54.746445 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-d623d87c-db90-40ab-9064-5551c89dc046 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:44:54.748000 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-d623d87c-db90-40ab-9064-5551c89dc046 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:44:54.825914 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 12/23] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 1 14:44:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 14:44:54.887022 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f8181d-45ba-4fe4-a409-faf0f2171e4a None None] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:44:54.887672 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f8181d-45ba-4fe4-a409-faf0f2171e4a None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:44:54.888158 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f8181d-45ba-4fe4-a409-faf0f2171e4a None None] PATH_INFO: `/` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:44:54.889974 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 12/24] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 1 14:44:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:44:54.894926 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ab5420-3714-47b9-8462-30e286fb0db2 None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:44:54.895218 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ab5420-3714-47b9-8462-30e286fb0db2 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:44:54.895496 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ab5420-3714-47b9-8462-30e286fb0db2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:44:54.923313 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-78ab5420-3714-47b9-8462-30e286fb0db2 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:44:54.924498 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-78ab5420-3714-47b9-8462-30e286fb0db2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:44:54.950200 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:44:54.974053 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b728733-991c-45bd-8ec0-9100fffcb219 None None] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:44:54.974435 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b728733-991c-45bd-8ec0-9100fffcb219 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:44:54.974654 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b728733-991c-45bd-8ec0-9100fffcb219 None None] PATH_INFO: `/` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:44:54.976177 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 13/25] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 1 14:44:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:44:54.981493 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5d0225-58f6-4be1-ba94-4d174fa42848 None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:44:54.982004 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5d0225-58f6-4be1-ba94-4d174fa42848 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:44:54.982520 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5d0225-58f6-4be1-ba94-4d174fa42848 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:44:55.004814 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 13/26] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 1 14:44:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 14:44:55.018148 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-bd5d0225-58f6-4be1-ba94-4d174fa42848 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:44:55.019625 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-bd5d0225-58f6-4be1-ba94-4d174fa42848 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:44:55.116252 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 14/27] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 1 14:44:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 14:44:55.239833 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599d0640-895f-4975-b46c-d8da864a53f7 None None] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:44:55.240131 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599d0640-895f-4975-b46c-d8da864a53f7 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:44:55.240309 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599d0640-895f-4975-b46c-d8da864a53f7 None None] PATH_INFO: `/` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:44:55.242290 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 14/28] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 1 14:44:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:44:55.247518 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8288b565-f662-47f1-8483-e262420960f8 None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:44:55.248037 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8288b565-f662-47f1-8483-e262420960f8 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:44:55.248512 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8288b565-f662-47f1-8483-e262420960f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:44:55.275198 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-8288b565-f662-47f1-8483-e262420960f8 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:44:55.276704 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-8288b565-f662-47f1-8483-e262420960f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:44:55.296564 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599d0640-895f-4975-b46c-d8da864a53f7 None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:44:55.397707 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 15/29] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 1 14:44:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 14:44:55.398878 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8288b565-f662-47f1-8483-e262420960f8 None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:44:55.450590 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:44:55.486647 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599d0640-895f-4975-b46c-d8da864a53f7 None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:44:55.538657 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da0dbe5-58fe-4f30-803f-45a45f9cd989 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:44:55.540157 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da0dbe5-58fe-4f30-803f-45a45f9cd989 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:44:55.541646 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da0dbe5-58fe-4f30-803f-45a45f9cd989 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:44:55.542116 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da0dbe5-58fe-4f30-803f-45a45f9cd989 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:44:55.563393 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da0dbe5-58fe-4f30-803f-45a45f9cd989 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:44:55.567121 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da0dbe5-58fe-4f30-803f-45a45f9cd989 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:44:55.598996 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application [None req-3da0dbe5-58fe-4f30-803f-45a45f9cd989 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 14:44:55.598996 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:44:55.598996 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:44:55.598996 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:44:55.598996 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:44:55.598996 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:44:55.598996 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:44:55.598996 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:44:55.598996 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:44:55.598996 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:44:55.598996 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:44:55.598996 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:44:55.598996 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 14:44:55.598996 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 14:44:55.598996 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 14:44:55.598996 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 14:44:55.599942 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:44:55.599942 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:44:55.599942 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:44:55.599942 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:44:55.599942 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:44:55.599942 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:44:55.599942 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:44:55.599942 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application with Lock( Sep 01 14:44:55.599942 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:44:55.599942 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:44:55.599942 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:44:55.599942 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:44:55.599942 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:44:55.599942 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:44:55.599942 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:44:55.599942 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:44:55.599942 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 14:44:55.601346 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 14:44:55.601346 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 14:44:55.601346 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Sep 01 14:44:55.605503 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 15/30] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 1 14:44:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 14:44:55.607388 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da0dbe5-58fe-4f30-803f-45a45f9cd989 None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:44:55.631795 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8288b565-f662-47f1-8483-e262420960f8 None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:44:55.633984 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:44:55.704249 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6bca1a-e679-4e69-83ff-6e71ec5bff70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:44:55.705262 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6bca1a-e679-4e69-83ff-6e71ec5bff70 None admin] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:44:55.706432 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6bca1a-e679-4e69-83ff-6e71ec5bff70 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:44:55.707076 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6bca1a-e679-4e69-83ff-6e71ec5bff70 None admin] PATH_INFO: `/v3/roles` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:44:55.709579 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6bca1a-e679-4e69-83ff-6e71ec5bff70 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:44:55.712668 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6bca1a-e679-4e69-83ff-6e71ec5bff70 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:44:55.776332 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da0dbe5-58fe-4f30-803f-45a45f9cd989 None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:44:55.811920 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee8d342-7050-4e04-b9fe-9b29bea25026 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:44:55.812554 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee8d342-7050-4e04-b9fe-9b29bea25026 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:44:55.812832 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee8d342-7050-4e04-b9fe-9b29bea25026 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:44:55.813105 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee8d342-7050-4e04-b9fe-9b29bea25026 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:44:55.827586 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee8d342-7050-4e04-b9fe-9b29bea25026 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 14:44:55.834106 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application [None req-7ee8d342-7050-4e04-b9fe-9b29bea25026 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 14:44:55.834106 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:44:55.834106 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:44:55.834106 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:44:55.834106 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:44:55.834106 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:44:55.834106 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:44:55.834106 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:44:55.834106 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:44:55.834106 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:44:55.834106 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:44:55.834106 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:44:55.834106 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 14:44:55.834106 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 14:44:55.834106 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 14:44:55.834106 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 14:44:55.835654 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 14:44:55.835654 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 14:44:55.835654 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 14:44:55.835654 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 14:44:55.835654 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:44:55.835654 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:44:55.835654 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:44:55.835654 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:44:55.835654 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:44:55.835654 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:44:55.835654 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:44:55.835654 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application with Lock( Sep 01 14:44:55.835654 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:44:55.835654 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:44:55.835654 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:44:55.835654 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:44:55.837360 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:44:55.837360 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:44:55.837360 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:44:55.837360 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:44:55.837360 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 14:44:55.837360 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 14:44:55.837360 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 14:44:55.837360 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 14:44:55.837360 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 14:44:55.837360 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 14:44:55.837360 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 14:44:55.837360 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Sep 01 14:44:55.837360 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 16/31] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Sep 1 14:44:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:44:55.837360 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee8d342-7050-4e04-b9fe-9b29bea25026 None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:44:55.852206 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 16/32] 10.222.0.33 () {60 vars in 1131 bytes} [Thu Sep 1 14:44:55 2022] POST /identity/v3/roles => generated 219 bytes in 454 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:44:55.853189 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6bca1a-e679-4e69-83ff-6e71ec5bff70 None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:44:55.971754 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:44:55.996593 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6bca1a-e679-4e69-83ff-6e71ec5bff70 None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:44:56.025595 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:44:56.040900 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca7e4fa-abb1-4264-8cb6-f0ebd672f012 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:44:56.041526 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca7e4fa-abb1-4264-8cb6-f0ebd672f012 None admin] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:44:56.041816 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca7e4fa-abb1-4264-8cb6-f0ebd672f012 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:44:56.042184 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca7e4fa-abb1-4264-8cb6-f0ebd672f012 None admin] PATH_INFO: `/v3/roles` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:44:56.042911 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca7e4fa-abb1-4264-8cb6-f0ebd672f012 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:44:56.044566 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca7e4fa-abb1-4264-8cb6-f0ebd672f012 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:44:56.062158 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 17/33] 10.222.0.33 () {60 vars in 1131 bytes} [Thu Sep 1 14:44:55 2022] POST /identity/v3/roles => generated 225 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:44:56.063044 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca7e4fa-abb1-4264-8cb6-f0ebd672f012 None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:44:56.064591 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee8d342-7050-4e04-b9fe-9b29bea25026 None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:44:56.112476 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b673109-bffe-45fd-8692-855cfd96d294 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:44:56.113161 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b673109-bffe-45fd-8692-855cfd96d294 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:44:56.113374 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b673109-bffe-45fd-8692-855cfd96d294 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:44:56.113649 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b673109-bffe-45fd-8692-855cfd96d294 None admin] PATH_INFO: `/v3/domains` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:44:56.117528 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b673109-bffe-45fd-8692-855cfd96d294 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81341) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:44:56.117704 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b673109-bffe-45fd-8692-855cfd96d294 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:44:56.119563 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b673109-bffe-45fd-8692-855cfd96d294 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:44:56.138266 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 17/34] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Sep 1 14:44:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:44:56.146298 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b673109-bffe-45fd-8692-855cfd96d294 None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:44:56.214161 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca7e4fa-abb1-4264-8cb6-f0ebd672f012 None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:44:56.257985 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4648c120-a61f-486c-84f2-f1befcd3d616 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:44:56.258661 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4648c120-a61f-486c-84f2-f1befcd3d616 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:44:56.258957 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4648c120-a61f-486c-84f2-f1befcd3d616 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:44:56.259231 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4648c120-a61f-486c-84f2-f1befcd3d616 None admin] PATH_INFO: `/v3/roles` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:44:56.260000 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4648c120-a61f-486c-84f2-f1befcd3d616 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:44:56.260313 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4648c120-a61f-486c-84f2-f1befcd3d616 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:44:56.261902 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4648c120-a61f-486c-84f2-f1befcd3d616 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:44:56.271534 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 18/35] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Sep 1 14:44:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:44:56.279454 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4648c120-a61f-486c-84f2-f1befcd3d616 None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:44:56.289874 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:44:56.324340 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b673109-bffe-45fd-8692-855cfd96d294 None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:44:56.359178 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:44:56.370473 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26818b2b-d44d-4231-8daa-b5c9647fa089 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:44:56.371089 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26818b2b-d44d-4231-8daa-b5c9647fa089 None admin] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:44:56.371375 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26818b2b-d44d-4231-8daa-b5c9647fa089 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:44:56.371636 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26818b2b-d44d-4231-8daa-b5c9647fa089 None admin] PATH_INFO: `/v3/projects` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:44:56.372310 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26818b2b-d44d-4231-8daa-b5c9647fa089 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:44:56.373795 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26818b2b-d44d-4231-8daa-b5c9647fa089 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:44:56.412867 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4648c120-a61f-486c-84f2-f1befcd3d616 None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:44:56.449707 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed80796-6aab-4dc0-9c18-431224a6efee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:44:56.450422 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed80796-6aab-4dc0-9c18-431224a6efee None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:44:56.450708 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed80796-6aab-4dc0-9c18-431224a6efee None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:44:56.450982 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed80796-6aab-4dc0-9c18-431224a6efee None admin] PATH_INFO: `/v3/users/abbabc07754b486599ec507190d24c97` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:44:56.452425 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed80796-6aab-4dc0-9c18-431224a6efee None admin] RBAC: Authorizing `identity:get_user(user_id=abbabc07754b486599ec507190d24c97)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:44:56.453877 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed80796-6aab-4dc0-9c18-431224a6efee None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:44:56.460222 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 19/36] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Sep 1 14:44:56 2022] GET /identity/v3/users/abbabc07754b486599ec507190d24c97 => generated 247 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:44:56.460222 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cae7a4-30b3-455d-9f29-9b14fda5c86e None None] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:44:56.460222 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cae7a4-30b3-455d-9f29-9b14fda5c86e None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:44:56.460222 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cae7a4-30b3-455d-9f29-9b14fda5c86e None None] PATH_INFO: `/` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:44:56.460222 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 20/37] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 1 14:44:56 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:44:56.461511 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cae7a4-30b3-455d-9f29-9b14fda5c86e None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:44:56.509583 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 18/38] 10.222.0.33 () {60 vars in 1141 bytes} [Thu Sep 1 14:44:56 2022] POST /identity/v3/projects => generated 301 bytes in 366 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 14:44:56.510870 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6e3b5a-361c-4956-a248-3ec857f0c5a8 None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:44:56.511164 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6e3b5a-361c-4956-a248-3ec857f0c5a8 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:44:56.511623 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6e3b5a-361c-4956-a248-3ec857f0c5a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:44:56.531356 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-5e6e3b5a-361c-4956-a248-3ec857f0c5a8 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:44:56.533076 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-5e6e3b5a-361c-4956-a248-3ec857f0c5a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:44:56.552719 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:44:56.598252 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cae7a4-30b3-455d-9f29-9b14fda5c86e None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:44:56.633482 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 19/39] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 1 14:44:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 14:44:56.644222 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc8445a-1c8b-47a6-a4c4-36402b9f1847 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:44:56.644222 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc8445a-1c8b-47a6-a4c4-36402b9f1847 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:44:56.644222 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc8445a-1c8b-47a6-a4c4-36402b9f1847 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:44:56.644222 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc8445a-1c8b-47a6-a4c4-36402b9f1847 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:44:56.644653 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc8445a-1c8b-47a6-a4c4-36402b9f1847 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:44:56.644653 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc8445a-1c8b-47a6-a4c4-36402b9f1847 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:44:56.648793 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 21/40] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 1 14:44:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:44:56.649983 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6e3b5a-361c-4956-a248-3ec857f0c5a8 None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:44:56.699953 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c523c7-562d-426d-8753-b3db1dd91edd None None] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:44:56.700402 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c523c7-562d-426d-8753-b3db1dd91edd None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:44:56.700777 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c523c7-562d-426d-8753-b3db1dd91edd None None] PATH_INFO: `/` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:44:56.702470 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 22/41] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 1 14:44:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 14:44:56.708358 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8364184c-841c-4caf-ad60-0957d1ace02f None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:44:56.708705 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8364184c-841c-4caf-ad60-0957d1ace02f None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:44:56.708996 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8364184c-841c-4caf-ad60-0957d1ace02f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:44:56.729952 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-8364184c-841c-4caf-ad60-0957d1ace02f None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:44:56.731596 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-8364184c-841c-4caf-ad60-0957d1ace02f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:44:56.814631 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 23/42] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 1 14:44:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 14:44:56.870569 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6e3b5a-361c-4956-a248-3ec857f0c5a8 None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:44:56.884824 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:44:56.905813 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8364184c-841c-4caf-ad60-0957d1ace02f None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:44:56.907467 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030c3628-dd42-48ab-a033-e6642334ab40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:44:56.908137 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030c3628-dd42-48ab-a033-e6642334ab40 None admin] REQUEST_METHOD: `PUT` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:44:56.908423 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030c3628-dd42-48ab-a033-e6642334ab40 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:44:56.908702 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030c3628-dd42-48ab-a033-e6642334ab40 None admin] PATH_INFO: `/v3/domains/default/users/abbabc07754b486599ec507190d24c97/roles/2a2adb769e55476b82e5014897e0a9c0` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:44:56.910688 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030c3628-dd42-48ab-a033-e6642334ab40 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=abbabc07754b486599ec507190d24c97, role_id=2a2adb769e55476b82e5014897e0a9c0)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:44:56.912451 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030c3628-dd42-48ab-a033-e6642334ab40 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:44:56.929660 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 20/43] 10.222.0.33 () {58 vars in 1361 bytes} [Thu Sep 1 14:44:56 2022] PUT /identity/v3/domains/default/users/abbabc07754b486599ec507190d24c97/roles/2a2adb769e55476b82e5014897e0a9c0 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:44:56.984437 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:44:57.019534 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8364184c-841c-4caf-ad60-0957d1ace02f None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:44:57.056342 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a76206-0693-4168-a74a-ae22b75ac6de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:44:57.056967 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a76206-0693-4168-a74a-ae22b75ac6de None admin] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:44:57.057301 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a76206-0693-4168-a74a-ae22b75ac6de None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:44:57.057579 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a76206-0693-4168-a74a-ae22b75ac6de None admin] PATH_INFO: `/v3/roles` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:44:57.058279 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a76206-0693-4168-a74a-ae22b75ac6de None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:44:57.059690 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a76206-0693-4168-a74a-ae22b75ac6de None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:44:57.077613 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 24/44] 10.222.0.33 () {60 vars in 1131 bytes} [Thu Sep 1 14:44:56 2022] POST /identity/v3/roles => generated 223 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 14:44:57.089872 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030c3628-dd42-48ab-a033-e6642334ab40 None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:44:57.206074 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030c3628-dd42-48ab-a033-e6642334ab40 None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:44:57.241700 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8758553c-a37b-4614-a3e6-033424e20523 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:44:57.242364 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8758553c-a37b-4614-a3e6-033424e20523 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:44:57.242654 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8758553c-a37b-4614-a3e6-033424e20523 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:44:57.242917 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8758553c-a37b-4614-a3e6-033424e20523 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:44:57.244028 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8758553c-a37b-4614-a3e6-033424e20523 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:44:57.245505 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8758553c-a37b-4614-a3e6-033424e20523 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:44:57.247317 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 21/45] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 1 14:44:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:44:57.252731 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a76206-0693-4168-a74a-ae22b75ac6de None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:44:57.354328 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a76206-0693-4168-a74a-ae22b75ac6de None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:44:57.383340 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:44:57.390764 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f979f0-cfe4-4e8f-bdab-c851e9e9d436 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:44:57.391384 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f979f0-cfe4-4e8f-bdab-c851e9e9d436 None admin] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:44:57.391665 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f979f0-cfe4-4e8f-bdab-c851e9e9d436 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:44:57.391938 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f979f0-cfe4-4e8f-bdab-c851e9e9d436 None admin] PATH_INFO: `/v3/projects` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:44:57.392601 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f979f0-cfe4-4e8f-bdab-c851e9e9d436 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:44:57.394037 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f979f0-cfe4-4e8f-bdab-c851e9e9d436 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:44:57.429612 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 25/46] 10.222.0.33 () {60 vars in 1141 bytes} [Thu Sep 1 14:44:57 2022] POST /identity/v3/projects => generated 312 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:44:58.862121 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe273586-7290-480c-9b3f-1682f2a4dd43 None None] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:44:58.862646 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe273586-7290-480c-9b3f-1682f2a4dd43 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:44:58.862751 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe273586-7290-480c-9b3f-1682f2a4dd43 None None] PATH_INFO: `/` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:44:58.864059 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 22/47] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 1 14:44:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:44:58.869390 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd937d24-5c22-4d74-b491-b066f6e44cc9 None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:44:58.869763 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd937d24-5c22-4d74-b491-b066f6e44cc9 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:44:58.869947 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd937d24-5c22-4d74-b491-b066f6e44cc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:44:58.888528 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-dd937d24-5c22-4d74-b491-b066f6e44cc9 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:44:58.889739 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-dd937d24-5c22-4d74-b491-b066f6e44cc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:44:58.955208 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 26/48] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 1 14:44:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 14:44:59.226720 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe273586-7290-480c-9b3f-1682f2a4dd43 None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:44:59.269195 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:44:59.300346 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b2ebf3-746e-4a17-8cae-282196c2dcbe None None] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:44:59.300635 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b2ebf3-746e-4a17-8cae-282196c2dcbe None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:44:59.300915 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b2ebf3-746e-4a17-8cae-282196c2dcbe None None] PATH_INFO: `/` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:44:59.302275 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 27/49] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 1 14:44:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 14:44:59.307526 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89deffc-191e-45c8-a02a-11411baadb69 None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:44:59.307812 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89deffc-191e-45c8-a02a-11411baadb69 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:44:59.308090 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89deffc-191e-45c8-a02a-11411baadb69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:44:59.326334 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-a89deffc-191e-45c8-a02a-11411baadb69 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:44:59.327556 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-a89deffc-191e-45c8-a02a-11411baadb69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:44:59.327748 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe273586-7290-480c-9b3f-1682f2a4dd43 None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:44:59.360464 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5d9ba5-b83e-4d50-ac14-1b86c11a3c38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:44:59.361118 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5d9ba5-b83e-4d50-ac14-1b86c11a3c38 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:44:59.361394 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5d9ba5-b83e-4d50-ac14-1b86c11a3c38 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:44:59.361665 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5d9ba5-b83e-4d50-ac14-1b86c11a3c38 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:44:59.371843 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5d9ba5-b83e-4d50-ac14-1b86c11a3c38 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 14:44:59.376603 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application [None req-ee5d9ba5-b83e-4d50-ac14-1b86c11a3c38 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 14:44:59.376603 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:44:59.376603 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:44:59.376603 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:44:59.376603 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:44:59.376603 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:44:59.376603 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:44:59.376603 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:44:59.376603 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:44:59.376603 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:44:59.376603 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:44:59.376603 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:44:59.376603 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 14:44:59.376603 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 14:44:59.376603 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 14:44:59.376603 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 14:44:59.377540 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 14:44:59.377540 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 14:44:59.377540 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 14:44:59.377540 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 14:44:59.377540 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:44:59.377540 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:44:59.377540 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:44:59.377540 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:44:59.377540 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:44:59.377540 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:44:59.377540 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:44:59.377540 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application with Lock( Sep 01 14:44:59.377540 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:44:59.377540 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:44:59.377540 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:44:59.377540 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:44:59.378388 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:44:59.378388 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:44:59.378388 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:44:59.378388 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:44:59.378388 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 14:44:59.378388 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 14:44:59.378388 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 14:44:59.378388 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 14:44:59.378388 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 14:44:59.378388 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 14:44:59.378388 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 14:44:59.378388 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Sep 01 14:44:59.378388 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 23/50] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Sep 1 14:44:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:44:59.384452 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5d9ba5-b83e-4d50-ac14-1b86c11a3c38 None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:44:59.393152 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 28/51] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 1 14:44:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 14:44:59.476581 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:44:59.479695 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5d9ba5-b83e-4d50-ac14-1b86c11a3c38 None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:44:59.513067 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1fb861-18ed-42fe-b531-4b0ad51eb72d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:44:59.513667 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1fb861-18ed-42fe-b531-4b0ad51eb72d None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:44:59.513948 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1fb861-18ed-42fe-b531-4b0ad51eb72d None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:44:59.514222 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1fb861-18ed-42fe-b531-4b0ad51eb72d None admin] PATH_INFO: `/v3/roles` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:44:59.514909 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1fb861-18ed-42fe-b531-4b0ad51eb72d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81341) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:44:59.515210 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1fb861-18ed-42fe-b531-4b0ad51eb72d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:44:59.516606 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1fb861-18ed-42fe-b531-4b0ad51eb72d None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:44:59.528096 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 24/52] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Sep 1 14:44:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 14:44:59.533537 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89deffc-191e-45c8-a02a-11411baadb69 None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:44:59.576741 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:44:59.632059 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89deffc-191e-45c8-a02a-11411baadb69 None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:44:59.658574 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1fb861-18ed-42fe-b531-4b0ad51eb72d None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:44:59.668294 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb317a5a-b55f-42ca-8ca5-691bf9879904 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:44:59.668927 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb317a5a-b55f-42ca-8ca5-691bf9879904 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:44:59.669205 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb317a5a-b55f-42ca-8ca5-691bf9879904 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:44:59.669475 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb317a5a-b55f-42ca-8ca5-691bf9879904 None admin] PATH_INFO: `/v3/users/abbabc07754b486599ec507190d24c97` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:44:59.670615 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb317a5a-b55f-42ca-8ca5-691bf9879904 None admin] RBAC: Authorizing `identity:get_user(user_id=abbabc07754b486599ec507190d24c97)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:44:59.672075 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb317a5a-b55f-42ca-8ca5-691bf9879904 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:44:59.673631 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 29/53] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Sep 1 14:44:59 2022] GET /identity/v3/users/abbabc07754b486599ec507190d24c97 => generated 247 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:44:59.679537 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb317a5a-b55f-42ca-8ca5-691bf9879904 None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:44:59.769789 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1fb861-18ed-42fe-b531-4b0ad51eb72d None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:44:59.804985 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5907db-8e91-4b27-86ae-c145992002e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:44:59.805629 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5907db-8e91-4b27-86ae-c145992002e8 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:44:59.805905 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5907db-8e91-4b27-86ae-c145992002e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:44:59.806181 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5907db-8e91-4b27-86ae-c145992002e8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:44:59.807338 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5907db-8e91-4b27-86ae-c145992002e8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:44:59.808826 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5907db-8e91-4b27-86ae-c145992002e8 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:44:59.810644 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 25/54] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 1 14:44:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 14:44:59.815881 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5907db-8e91-4b27-86ae-c145992002e8 None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:44:59.845742 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:44:59.882084 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb317a5a-b55f-42ca-8ca5-691bf9879904 None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:44:59.892310 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:44:59.915753 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5907db-8e91-4b27-86ae-c145992002e8 None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:44:59.916179 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409a6ab4-bf1b-4c0c-adae-26cdaa52ffe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:44:59.916550 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409a6ab4-bf1b-4c0c-adae-26cdaa52ffe9 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:44:59.916830 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409a6ab4-bf1b-4c0c-adae-26cdaa52ffe9 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:44:59.917099 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409a6ab4-bf1b-4c0c-adae-26cdaa52ffe9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:44:59.918193 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409a6ab4-bf1b-4c0c-adae-26cdaa52ffe9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:44:59.919666 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409a6ab4-bf1b-4c0c-adae-26cdaa52ffe9 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:44:59.921214 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 30/55] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 1 14:44:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 14:44:59.927000 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409a6ab4-bf1b-4c0c-adae-26cdaa52ffe9 None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:44:59.949051 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c33a056-a802-4b63-a3e9-3460962b172d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:44:59.949656 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c33a056-a802-4b63-a3e9-3460962b172d None admin] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:44:59.949939 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c33a056-a802-4b63-a3e9-3460962b172d None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:44:59.950212 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c33a056-a802-4b63-a3e9-3460962b172d None admin] PATH_INFO: `/v3/projects` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:44:59.950894 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c33a056-a802-4b63-a3e9-3460962b172d None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:44:59.952346 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c33a056-a802-4b63-a3e9-3460962b172d None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:44:59.982221 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 26/56] 10.222.0.33 () {60 vars in 1140 bytes} [Thu Sep 1 14:44:59 2022] POST /identity/v3/projects => generated 298 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 14:45:00.030076 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409a6ab4-bf1b-4c0c-adae-26cdaa52ffe9 None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:00.065858 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2381e2f-5df4-4519-b192-d3bca8c0a40e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:00.066442 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2381e2f-5df4-4519-b192-d3bca8c0a40e None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:00.066751 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2381e2f-5df4-4519-b192-d3bca8c0a40e None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:00.067030 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2381e2f-5df4-4519-b192-d3bca8c0a40e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:00.067751 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2381e2f-5df4-4519-b192-d3bca8c0a40e None admin] RBAC: Adding query filter params (role.id=2a2adb769e55476b82e5014897e0a9c0, scope.domain.id=default, user.id=abbabc07754b486599ec507190d24c97) {{(pid=81342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:45:00.068061 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2381e2f-5df4-4519-b192-d3bca8c0a40e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:00.069533 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2381e2f-5df4-4519-b192-d3bca8c0a40e None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:00.079163 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 31/57] 10.222.0.33 () {56 vars in 1322 bytes} [Thu Sep 1 14:44:59 2022] GET /identity/v3/role_assignments?user.id=abbabc07754b486599ec507190d24c97&scope.domain.id=default&role.id=2a2adb769e55476b82e5014897e0a9c0 => generated 530 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 14:45:01.860940 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2533c31f-c640-4137-bad9-6138edc7c389 None None] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:01.861473 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2533c31f-c640-4137-bad9-6138edc7c389 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:01.861473 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2533c31f-c640-4137-bad9-6138edc7c389 None None] PATH_INFO: `/` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:01.862856 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 27/58] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 1 14:45:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:45:01.867788 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d8f375-8269-413c-a517-09e50ecc48ac None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:01.868188 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d8f375-8269-413c-a517-09e50ecc48ac None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:01.868347 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d8f375-8269-413c-a517-09e50ecc48ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:01.886944 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-f9d8f375-8269-413c-a517-09e50ecc48ac None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:45:01.888148 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-f9d8f375-8269-413c-a517-09e50ecc48ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:45:01.922311 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:01.953295 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 32/59] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 1 14:45:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 14:45:02.223060 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2533c31f-c640-4137-bad9-6138edc7c389 None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:02.302193 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:02.322965 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2533c31f-c640-4137-bad9-6138edc7c389 None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:02.355129 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315a23ca-dbeb-4be2-b1f8-da9ea1311502 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:02.355751 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315a23ca-dbeb-4be2-b1f8-da9ea1311502 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:02.356035 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315a23ca-dbeb-4be2-b1f8-da9ea1311502 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:02.356315 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315a23ca-dbeb-4be2-b1f8-da9ea1311502 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:02.357432 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315a23ca-dbeb-4be2-b1f8-da9ea1311502 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:02.358936 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315a23ca-dbeb-4be2-b1f8-da9ea1311502 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:02.360484 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 28/60] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 1 14:45:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:45:02.365745 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d8f375-8269-413c-a517-09e50ecc48ac None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:02.462431 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d8f375-8269-413c-a517-09e50ecc48ac None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:02.495234 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d624746-48dd-427f-8e34-f9757a1f6133 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:02.495781 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d624746-48dd-427f-8e34-f9757a1f6133 None admin] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:02.496067 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d624746-48dd-427f-8e34-f9757a1f6133 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:02.496344 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d624746-48dd-427f-8e34-f9757a1f6133 None admin] PATH_INFO: `/v3/users` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:02.496999 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d624746-48dd-427f-8e34-f9757a1f6133 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:02.498543 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d624746-48dd-427f-8e34-f9757a1f6133 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:02.525615 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 33/61] 10.222.0.33 () {60 vars in 1132 bytes} [Thu Sep 1 14:45:02 2022] POST /identity/v3/users => generated 275 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:45:05.564587 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3062cf-4504-47d9-ade4-0bf4d89e7da7 None None] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:05.565554 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3062cf-4504-47d9-ade4-0bf4d89e7da7 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:05.566041 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3062cf-4504-47d9-ade4-0bf4d89e7da7 None None] PATH_INFO: `/` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:05.567612 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 29/62] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 1 14:45:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:45:05.573571 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da836cf-ec77-4cca-9818-22387235d6c3 None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:05.574409 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da836cf-ec77-4cca-9818-22387235d6c3 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:05.574884 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da836cf-ec77-4cca-9818-22387235d6c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:05.599215 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-4da836cf-ec77-4cca-9818-22387235d6c3 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:45:05.600938 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-4da836cf-ec77-4cca-9818-22387235d6c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:45:05.628304 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:05.686490 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 34/63] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 1 14:45:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 14:45:05.757289 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b2691d-520a-4e87-b47f-2c2461fc0273 None None] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:05.757586 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b2691d-520a-4e87-b47f-2c2461fc0273 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:05.757856 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b2691d-520a-4e87-b47f-2c2461fc0273 None None] PATH_INFO: `/` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:05.759359 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 30/64] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 1 14:45:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:45:05.768068 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be24ad5-1092-4228-8163-55563e526021 None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:05.768363 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be24ad5-1092-4228-8163-55563e526021 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:05.768643 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be24ad5-1092-4228-8163-55563e526021 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:05.799254 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-7be24ad5-1092-4228-8163-55563e526021 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:45:05.800513 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-7be24ad5-1092-4228-8163-55563e526021 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:45:05.848846 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27d463c-b960-490b-ab34-35e8c6c63e85 None None] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:05.849112 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27d463c-b960-490b-ab34-35e8c6c63e85 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:05.849324 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27d463c-b960-490b-ab34-35e8c6c63e85 None None] PATH_INFO: `/` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:05.850729 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 31/65] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 1 14:45:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:45:05.863373 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35dc6b9-52ed-4d1f-beab-d78c70d7bdc1 None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:05.863660 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35dc6b9-52ed-4d1f-beab-d78c70d7bdc1 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:05.863939 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35dc6b9-52ed-4d1f-beab-d78c70d7bdc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:05.886466 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-d35dc6b9-52ed-4d1f-beab-d78c70d7bdc1 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:45:05.888032 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-d35dc6b9-52ed-4d1f-beab-d78c70d7bdc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:45:05.899696 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 35/66] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 1 14:45:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 14:45:05.901152 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1381ae-faa9-450d-8771-bed79c9060bd None None] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:05.901573 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1381ae-faa9-450d-8771-bed79c9060bd None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:05.901978 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1381ae-faa9-450d-8771-bed79c9060bd None None] PATH_INFO: `/` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:05.911591 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 36/67] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 1 14:45:05 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:45:05.917059 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a8f2df-492a-47fb-bacb-865bd06ce3d4 None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:05.917672 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a8f2df-492a-47fb-bacb-865bd06ce3d4 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:05.918137 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a8f2df-492a-47fb-bacb-865bd06ce3d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:05.948338 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-d8a8f2df-492a-47fb-bacb-865bd06ce3d4 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:45:05.949815 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-d8a8f2df-492a-47fb-bacb-865bd06ce3d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:45:05.966596 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 32/68] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 1 14:45:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 14:45:06.008778 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35dc6b9-52ed-4d1f-beab-d78c70d7bdc1 None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:06.084870 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 37/69] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 1 14:45:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 14:45:06.104149 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:06.126763 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35dc6b9-52ed-4d1f-beab-d78c70d7bdc1 None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:06.163116 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb1927f-59d7-45c3-81b8-2fd222ee4ccd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:06.164050 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb1927f-59d7-45c3-81b8-2fd222ee4ccd None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:06.164477 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb1927f-59d7-45c3-81b8-2fd222ee4ccd None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:06.164894 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb1927f-59d7-45c3-81b8-2fd222ee4ccd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:06.177032 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb1927f-59d7-45c3-81b8-2fd222ee4ccd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 14:45:06.182688 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application [None req-ddb1927f-59d7-45c3-81b8-2fd222ee4ccd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 14:45:06.182688 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:45:06.182688 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:45:06.182688 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:45:06.182688 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:45:06.182688 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:45:06.182688 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:45:06.182688 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:45:06.182688 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:45:06.182688 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:45:06.182688 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:45:06.182688 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:45:06.182688 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 14:45:06.182688 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 14:45:06.182688 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 14:45:06.182688 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 14:45:06.183756 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 14:45:06.183756 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 14:45:06.183756 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 14:45:06.183756 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 14:45:06.183756 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:45:06.183756 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:45:06.183756 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:45:06.183756 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:45:06.183756 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:45:06.183756 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:45:06.183756 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:45:06.183756 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application with Lock( Sep 01 14:45:06.183756 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:45:06.183756 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:45:06.183756 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:45:06.183756 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:45:06.184718 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:45:06.184718 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:45:06.184718 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:45:06.184718 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:45:06.184718 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 14:45:06.184718 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 14:45:06.184718 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 14:45:06.184718 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 14:45:06.184718 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 14:45:06.184718 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 14:45:06.184718 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 14:45:06.184718 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Sep 01 14:45:06.186544 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 33/70] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Sep 1 14:45:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 14:45:06.198997 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a8f2df-492a-47fb-bacb-865bd06ce3d4 None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:06.234778 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:06.246071 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb1927f-59d7-45c3-81b8-2fd222ee4ccd None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:06.358463 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb1927f-59d7-45c3-81b8-2fd222ee4ccd None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:06.358903 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a8f2df-492a-47fb-bacb-865bd06ce3d4 None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:06.395255 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bc4ddd-b2ed-4005-8ee1-70f4e6e92c3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:06.395476 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5450018f-ea46-43bd-a514-2ff94d13d88c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:06.395893 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bc4ddd-b2ed-4005-8ee1-70f4e6e92c3d None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:06.395997 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5450018f-ea46-43bd-a514-2ff94d13d88c None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:06.396177 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bc4ddd-b2ed-4005-8ee1-70f4e6e92c3d None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:06.396264 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5450018f-ea46-43bd-a514-2ff94d13d88c None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:06.396454 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bc4ddd-b2ed-4005-8ee1-70f4e6e92c3d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:06.396540 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5450018f-ea46-43bd-a514-2ff94d13d88c None admin] PATH_INFO: `/v3/roles` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:06.397164 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5450018f-ea46-43bd-a514-2ff94d13d88c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:45:06.397473 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5450018f-ea46-43bd-a514-2ff94d13d88c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:06.399129 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5450018f-ea46-43bd-a514-2ff94d13d88c None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:06.403975 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:06.407109 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 38/71] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Sep 1 14:45:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:45:06.407453 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bc4ddd-b2ed-4005-8ee1-70f4e6e92c3d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 14:45:06.407907 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5450018f-ea46-43bd-a514-2ff94d13d88c None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:06.413528 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application [None req-35bc4ddd-b2ed-4005-8ee1-70f4e6e92c3d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 14:45:06.413528 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:45:06.413528 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:45:06.413528 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:45:06.413528 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:45:06.413528 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:45:06.413528 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:45:06.413528 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:45:06.413528 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:45:06.413528 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:45:06.413528 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:45:06.413528 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:45:06.413528 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 14:45:06.413528 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 14:45:06.413528 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 14:45:06.413528 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 14:45:06.414594 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 14:45:06.414594 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 14:45:06.414594 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 14:45:06.414594 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 14:45:06.414594 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:45:06.414594 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:45:06.414594 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:45:06.414594 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:45:06.414594 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:45:06.414594 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:45:06.414594 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:45:06.414594 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application with Lock( Sep 01 14:45:06.414594 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:45:06.414594 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:45:06.414594 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:45:06.414594 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:45:06.419470 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:45:06.419470 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:45:06.419470 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:45:06.419470 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:45:06.419470 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 14:45:06.419470 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 14:45:06.419470 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 14:45:06.419470 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 14:45:06.419470 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 14:45:06.419470 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 14:45:06.419470 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 14:45:06.419470 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Sep 01 14:45:06.419470 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 34/72] 10.222.0.33 () {56 vars in 1114 bytes} [Thu Sep 1 14:45:06 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:45:06.419470 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb20f02c-f0eb-4bca-9d0f-2a38fd80e98e None None] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:06.420220 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb20f02c-f0eb-4bca-9d0f-2a38fd80e98e None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:06.420220 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb20f02c-f0eb-4bca-9d0f-2a38fd80e98e None None] PATH_INFO: `/` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:06.420220 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 35/73] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 1 14:45:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:45:06.423248 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb20f02c-f0eb-4bca-9d0f-2a38fd80e98e None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:06.492858 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:06.512694 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5450018f-ea46-43bd-a514-2ff94d13d88c None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:06.528163 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb20f02c-f0eb-4bca-9d0f-2a38fd80e98e None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:06.546177 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe59458-e869-419f-89cb-14aab22377e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:06.546832 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe59458-e869-419f-89cb-14aab22377e7 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:06.547119 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe59458-e869-419f-89cb-14aab22377e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:06.547392 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe59458-e869-419f-89cb-14aab22377e7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:06.558206 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe59458-e869-419f-89cb-14aab22377e7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 14:45:06.562744 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ffb407-c86e-4ef9-b14f-120bce913569 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:06.563076 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application [None req-bbe59458-e869-419f-89cb-14aab22377e7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 14:45:06.563076 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:45:06.563076 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:45:06.563076 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:45:06.563076 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:45:06.563076 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:45:06.563076 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:45:06.563076 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:45:06.563076 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:45:06.563076 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:45:06.563076 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:45:06.563076 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:45:06.563076 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 14:45:06.563076 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 14:45:06.563076 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 14:45:06.563076 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 14:45:06.564075 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 14:45:06.564075 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 14:45:06.564075 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 14:45:06.564075 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 14:45:06.564075 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:45:06.564075 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:45:06.564075 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:45:06.564075 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:45:06.564075 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:45:06.564075 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:45:06.564075 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:45:06.564075 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application with Lock( Sep 01 14:45:06.564075 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:45:06.564075 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:45:06.564075 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:45:06.564075 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:45:06.564978 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:45:06.564978 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:45:06.564978 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:45:06.564978 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:45:06.564978 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 14:45:06.564978 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 14:45:06.564978 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 14:45:06.564978 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 14:45:06.564978 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 14:45:06.564978 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 14:45:06.564978 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 14:45:06.564978 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Sep 01 14:45:06.565688 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ffb407-c86e-4ef9-b14f-120bce913569 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:06.565688 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ffb407-c86e-4ef9-b14f-120bce913569 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:06.565688 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ffb407-c86e-4ef9-b14f-120bce913569 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:06.566264 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 39/74] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Sep 1 14:45:06 2022] GET /identity/v3/roles/member => generated 84 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:45:06.566264 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe59458-e869-419f-89cb-14aab22377e7 None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:06.577252 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ffb407-c86e-4ef9-b14f-120bce913569 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 14:45:06.582131 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application [None req-31ffb407-c86e-4ef9-b14f-120bce913569 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 14:45:06.582131 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:45:06.582131 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:45:06.582131 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:45:06.582131 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:45:06.582131 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:45:06.582131 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:45:06.582131 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:45:06.582131 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:45:06.582131 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:45:06.582131 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:45:06.582131 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:45:06.582131 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 14:45:06.582131 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 14:45:06.582131 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 14:45:06.582131 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 14:45:06.583006 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 14:45:06.583006 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 14:45:06.583006 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 14:45:06.583006 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 14:45:06.583006 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:45:06.583006 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:45:06.583006 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:45:06.583006 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:45:06.583006 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:45:06.583006 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:45:06.583006 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:45:06.583006 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application with Lock( Sep 01 14:45:06.583006 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:45:06.583006 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:45:06.583006 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:45:06.583006 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:45:06.583854 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:45:06.583854 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:45:06.583854 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:45:06.583854 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:45:06.583854 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 14:45:06.583854 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 14:45:06.583854 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 14:45:06.583854 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 14:45:06.583854 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 14:45:06.583854 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 14:45:06.583854 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 14:45:06.583854 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Sep 01 14:45:06.583854 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 36/75] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Sep 1 14:45:06 2022] GET /identity/v3/roles/member => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:45:06.584508 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ffb407-c86e-4ef9-b14f-120bce913569 None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:06.644148 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:06.671991 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe59458-e869-419f-89cb-14aab22377e7 None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:06.688093 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ffb407-c86e-4ef9-b14f-120bce913569 None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:06.705415 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7289b8f-2251-4157-a71f-67c6d8e8a476 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:06.706047 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7289b8f-2251-4157-a71f-67c6d8e8a476 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:06.706463 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7289b8f-2251-4157-a71f-67c6d8e8a476 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:06.706741 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7289b8f-2251-4157-a71f-67c6d8e8a476 None admin] PATH_INFO: `/v3/users/abbabc07754b486599ec507190d24c97` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:06.707892 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7289b8f-2251-4157-a71f-67c6d8e8a476 None admin] RBAC: Authorizing `identity:get_user(user_id=abbabc07754b486599ec507190d24c97)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:06.709360 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7289b8f-2251-4157-a71f-67c6d8e8a476 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:06.711483 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 40/76] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Sep 1 14:45:06 2022] GET /identity/v3/users/abbabc07754b486599ec507190d24c97 => generated 247 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:45:06.712572 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa32e19-f36b-473b-aef4-f00ff18a21d5 None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:06.712853 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa32e19-f36b-473b-aef4-f00ff18a21d5 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:06.713128 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa32e19-f36b-473b-aef4-f00ff18a21d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:06.727494 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8691724d-7150-446c-9696-adec8a3be3d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:06.728182 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8691724d-7150-446c-9696-adec8a3be3d7 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:06.728472 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8691724d-7150-446c-9696-adec8a3be3d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:06.728750 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8691724d-7150-446c-9696-adec8a3be3d7 None admin] PATH_INFO: `/v3/roles` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:06.729426 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8691724d-7150-446c-9696-adec8a3be3d7 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81341) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:45:06.729731 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8691724d-7150-446c-9696-adec8a3be3d7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:06.731174 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8691724d-7150-446c-9696-adec8a3be3d7 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:06.731349 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-0fa32e19-f36b-473b-aef4-f00ff18a21d5 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:45:06.732550 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-0fa32e19-f36b-473b-aef4-f00ff18a21d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:45:06.734510 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:06.739369 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 37/77] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Sep 1 14:45:06 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:45:06.740165 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8691724d-7150-446c-9696-adec8a3be3d7 None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:06.798519 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 41/78] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 1 14:45:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 14:45:06.799353 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa32e19-f36b-473b-aef4-f00ff18a21d5 None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:06.842308 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8691724d-7150-446c-9696-adec8a3be3d7 None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:06.864727 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:06.877423 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b638cc1a-3d96-49fe-916f-68ba06a5b7e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:06.878043 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b638cc1a-3d96-49fe-916f-68ba06a5b7e1 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:06.878360 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b638cc1a-3d96-49fe-916f-68ba06a5b7e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:06.878638 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b638cc1a-3d96-49fe-916f-68ba06a5b7e1 None admin] PATH_INFO: `/v3/roles` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:06.879333 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b638cc1a-3d96-49fe-916f-68ba06a5b7e1 None admin] RBAC: Adding query filter params (name=member) {{(pid=81341) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:45:06.879639 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b638cc1a-3d96-49fe-916f-68ba06a5b7e1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:06.881043 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b638cc1a-3d96-49fe-916f-68ba06a5b7e1 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:06.889509 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 38/79] 10.222.0.33 () {56 vars in 1101 bytes} [Thu Sep 1 14:45:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:45:06.890146 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b638cc1a-3d96-49fe-916f-68ba06a5b7e1 None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:06.904873 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa32e19-f36b-473b-aef4-f00ff18a21d5 None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:06.940924 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e4e5de-3304-4354-934b-60f27e9b53a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:06.941583 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e4e5de-3304-4354-934b-60f27e9b53a0 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:06.941864 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e4e5de-3304-4354-934b-60f27e9b53a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:06.942132 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e4e5de-3304-4354-934b-60f27e9b53a0 None admin] PATH_INFO: `/v3/roles` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:06.942859 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e4e5de-3304-4354-934b-60f27e9b53a0 None admin] RBAC: Adding query filter params (name=member) {{(pid=81342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:45:06.943164 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e4e5de-3304-4354-934b-60f27e9b53a0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:06.944575 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e4e5de-3304-4354-934b-60f27e9b53a0 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:06.949837 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:06.954082 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 42/80] 10.222.0.33 () {56 vars in 1101 bytes} [Thu Sep 1 14:45:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:45:06.954920 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e4e5de-3304-4354-934b-60f27e9b53a0 None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:06.996344 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b638cc1a-3d96-49fe-916f-68ba06a5b7e1 None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:07.033748 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211bebb6-bfeb-46f2-ac5e-efc48e739ddb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:07.034385 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211bebb6-bfeb-46f2-ac5e-efc48e739ddb None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:07.034665 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211bebb6-bfeb-46f2-ac5e-efc48e739ddb None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:07.034938 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211bebb6-bfeb-46f2-ac5e-efc48e739ddb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:07.062921 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e4e5de-3304-4354-934b-60f27e9b53a0 None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:07.088057 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:07.099371 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263a0a85-fff3-4309-9ff6-975c5f156668 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:07.099999 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263a0a85-fff3-4309-9ff6-975c5f156668 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:07.100278 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263a0a85-fff3-4309-9ff6-975c5f156668 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:07.100548 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263a0a85-fff3-4309-9ff6-975c5f156668 None admin] PATH_INFO: `/v3/users/4b65298713a24cada6ba75d786076e93` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:07.104170 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211bebb6-bfeb-46f2-ac5e-efc48e739ddb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:07.105682 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211bebb6-bfeb-46f2-ac5e-efc48e739ddb None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:07.117128 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263a0a85-fff3-4309-9ff6-975c5f156668 None admin] RBAC: Authorizing `identity:get_user(user_id=4b65298713a24cada6ba75d786076e93)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:07.118610 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263a0a85-fff3-4309-9ff6-975c5f156668 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:07.120220 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 43/81] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Sep 1 14:45:06 2022] GET /identity/v3/users/4b65298713a24cada6ba75d786076e93 => generated 275 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:45:07.120980 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263a0a85-fff3-4309-9ff6-975c5f156668 None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:07.181782 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:07.203538 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 39/82] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Sep 1 14:45:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 14:45:07.204375 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211bebb6-bfeb-46f2-ac5e-efc48e739ddb None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:07.228426 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263a0a85-fff3-4309-9ff6-975c5f156668 None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:07.264557 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b89e8a-538d-4fb7-adb8-b154e8c1f763 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:07.265180 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b89e8a-538d-4fb7-adb8-b154e8c1f763 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:07.265460 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b89e8a-538d-4fb7-adb8-b154e8c1f763 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:07.265728 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b89e8a-538d-4fb7-adb8-b154e8c1f763 None admin] PATH_INFO: `/v3/users/4b65298713a24cada6ba75d786076e93` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:07.266834 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b89e8a-538d-4fb7-adb8-b154e8c1f763 None admin] RBAC: Authorizing `identity:get_user(user_id=4b65298713a24cada6ba75d786076e93)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:07.268294 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b89e8a-538d-4fb7-adb8-b154e8c1f763 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:07.269849 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 44/83] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Sep 1 14:45:07 2022] GET /identity/v3/users/4b65298713a24cada6ba75d786076e93 => generated 275 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:45:07.270630 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b89e8a-538d-4fb7-adb8-b154e8c1f763 None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:07.291192 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:07.306520 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211bebb6-bfeb-46f2-ac5e-efc48e739ddb None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:07.340998 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6e85c3-2a68-4a6f-b529-7f0c3a8877de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:07.340998 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6e85c3-2a68-4a6f-b529-7f0c3a8877de None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:07.341244 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6e85c3-2a68-4a6f-b529-7f0c3a8877de None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:07.341514 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6e85c3-2a68-4a6f-b529-7f0c3a8877de None admin] PATH_INFO: `/v3/users/4b65298713a24cada6ba75d786076e93` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:07.342665 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6e85c3-2a68-4a6f-b529-7f0c3a8877de None admin] RBAC: Authorizing `identity:get_user(user_id=4b65298713a24cada6ba75d786076e93)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:07.344132 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6e85c3-2a68-4a6f-b529-7f0c3a8877de None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:07.345694 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 40/84] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Sep 1 14:45:07 2022] GET /identity/v3/users/4b65298713a24cada6ba75d786076e93 => generated 275 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:45:07.346473 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6e85c3-2a68-4a6f-b529-7f0c3a8877de None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:07.369566 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b89e8a-538d-4fb7-adb8-b154e8c1f763 None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:07.394507 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:07.404096 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a24f84-915e-4093-9f3f-44b55bd65a08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:07.404723 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a24f84-915e-4093-9f3f-44b55bd65a08 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:07.405007 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a24f84-915e-4093-9f3f-44b55bd65a08 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:07.405280 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a24f84-915e-4093-9f3f-44b55bd65a08 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:07.406403 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a24f84-915e-4093-9f3f-44b55bd65a08 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:07.407863 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a24f84-915e-4093-9f3f-44b55bd65a08 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:07.409425 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 45/85] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 1 14:45:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:45:07.410182 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a24f84-915e-4093-9f3f-44b55bd65a08 None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:07.448569 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6e85c3-2a68-4a6f-b529-7f0c3a8877de None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:07.485197 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26c098e-0138-4d7c-a387-7dbb270ff89a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:07.485802 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26c098e-0138-4d7c-a387-7dbb270ff89a None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:07.486092 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26c098e-0138-4d7c-a387-7dbb270ff89a None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:07.486400 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26c098e-0138-4d7c-a387-7dbb270ff89a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:07.507961 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:07.509179 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a24f84-915e-4093-9f3f-44b55bd65a08 None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:07.541869 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a100e8b-0bfe-4d5a-93f2-9b7f8823d519 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:07.542536 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a100e8b-0bfe-4d5a-93f2-9b7f8823d519 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:07.542821 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a100e8b-0bfe-4d5a-93f2-9b7f8823d519 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:07.543102 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a100e8b-0bfe-4d5a-93f2-9b7f8823d519 None admin] PATH_INFO: `/v3/projects/1f696f55becc4e879b6d1bac9d93d055` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:07.544224 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a100e8b-0bfe-4d5a-93f2-9b7f8823d519 None admin] RBAC: Authorizing `identity:get_project(project_id=1f696f55becc4e879b6d1bac9d93d055)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:07.545662 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a100e8b-0bfe-4d5a-93f2-9b7f8823d519 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:07.547312 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 46/86] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Sep 1 14:45:07 2022] GET /identity/v3/projects/1f696f55becc4e879b6d1bac9d93d055 => generated 298 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:45:07.551500 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a100e8b-0bfe-4d5a-93f2-9b7f8823d519 None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:07.553257 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26c098e-0138-4d7c-a387-7dbb270ff89a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:07.555442 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26c098e-0138-4d7c-a387-7dbb270ff89a None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:07.612320 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:07.651139 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a100e8b-0bfe-4d5a-93f2-9b7f8823d519 None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:07.652881 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 41/87] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Sep 1 14:45:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 14:45:07.653733 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26c098e-0138-4d7c-a387-7dbb270ff89a None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:07.688371 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc2dab6-41ea-49f8-97b6-7de281e38b89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:07.688988 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc2dab6-41ea-49f8-97b6-7de281e38b89 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:07.689274 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc2dab6-41ea-49f8-97b6-7de281e38b89 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:07.689548 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc2dab6-41ea-49f8-97b6-7de281e38b89 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:07.710300 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:07.756292 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26c098e-0138-4d7c-a387-7dbb270ff89a None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:07.757417 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc2dab6-41ea-49f8-97b6-7de281e38b89 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:07.758942 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc2dab6-41ea-49f8-97b6-7de281e38b89 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:07.788946 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8b8473-f655-461f-b7bb-841517a2bbad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:07.789548 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8b8473-f655-461f-b7bb-841517a2bbad None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:07.789829 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8b8473-f655-461f-b7bb-841517a2bbad None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:07.790102 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8b8473-f655-461f-b7bb-841517a2bbad None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:07.816633 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:07.855897 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8b8473-f655-461f-b7bb-841517a2bbad None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:07.856539 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 47/88] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Sep 1 14:45:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 14:45:07.857340 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc2dab6-41ea-49f8-97b6-7de281e38b89 None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:07.857439 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8b8473-f655-461f-b7bb-841517a2bbad None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:07.910032 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:07.953740 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc2dab6-41ea-49f8-97b6-7de281e38b89 None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:07.958963 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 42/89] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Sep 1 14:45:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 14:45:07.959808 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8b8473-f655-461f-b7bb-841517a2bbad None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:07.986905 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb6168e-28ac-4253-9c24-18d150f58891 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:07.987446 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb6168e-28ac-4253-9c24-18d150f58891 None admin] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:07.987728 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb6168e-28ac-4253-9c24-18d150f58891 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:07.988000 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb6168e-28ac-4253-9c24-18d150f58891 None admin] PATH_INFO: `/v3/users` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:07.988649 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb6168e-28ac-4253-9c24-18d150f58891 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:07.990105 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb6168e-28ac-4253-9c24-18d150f58891 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:08.014265 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 48/90] 10.222.0.33 () {60 vars in 1132 bytes} [Thu Sep 1 14:45:07 2022] POST /identity/v3/users => generated 289 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:45:08.015320 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb6168e-28ac-4253-9c24-18d150f58891 None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:08.039216 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:08.075015 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8b8473-f655-461f-b7bb-841517a2bbad None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:08.114652 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f09f5c7-b58a-492d-a09b-2fac02fe6d6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:08.115708 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f09f5c7-b58a-492d-a09b-2fac02fe6d6c None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:08.116990 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f09f5c7-b58a-492d-a09b-2fac02fe6d6c None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:08.117469 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f09f5c7-b58a-492d-a09b-2fac02fe6d6c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:08.118463 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f09f5c7-b58a-492d-a09b-2fac02fe6d6c None admin] RBAC: Adding query filter params (role.id=2a2adb769e55476b82e5014897e0a9c0, scope.project.id=1f696f55becc4e879b6d1bac9d93d055, user.id=abbabc07754b486599ec507190d24c97) {{(pid=81341) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:45:08.118957 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f09f5c7-b58a-492d-a09b-2fac02fe6d6c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:08.121519 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f09f5c7-b58a-492d-a09b-2fac02fe6d6c None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:08.129043 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 43/91] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Sep 1 14:45:07 2022] GET /identity/v3/role_assignments?user.id=abbabc07754b486599ec507190d24c97&scope.project.id=1f696f55becc4e879b6d1bac9d93d055&role.id=2a2adb769e55476b82e5014897e0a9c0 => generated 259 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:45:08.129814 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f09f5c7-b58a-492d-a09b-2fac02fe6d6c None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:08.132605 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb6168e-28ac-4253-9c24-18d150f58891 None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:08.163978 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:08.179553 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7add88-2f82-4b87-8bda-532a481cbbf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:08.180220 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7add88-2f82-4b87-8bda-532a481cbbf7 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:08.180510 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7add88-2f82-4b87-8bda-532a481cbbf7 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:08.180790 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7add88-2f82-4b87-8bda-532a481cbbf7 None admin] PATH_INFO: `/v3/projects/1f696f55becc4e879b6d1bac9d93d055` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:08.182047 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7add88-2f82-4b87-8bda-532a481cbbf7 None admin] RBAC: Authorizing `identity:get_project(project_id=1f696f55becc4e879b6d1bac9d93d055)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:08.184995 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7add88-2f82-4b87-8bda-532a481cbbf7 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:08.185449 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 49/92] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Sep 1 14:45:08 2022] GET /identity/v3/projects/1f696f55becc4e879b6d1bac9d93d055 => generated 298 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:45:08.191298 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7add88-2f82-4b87-8bda-532a481cbbf7 None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:08.285922 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f09f5c7-b58a-492d-a09b-2fac02fe6d6c None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:08.305283 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:08.332651 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7add88-2f82-4b87-8bda-532a481cbbf7 None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:08.337040 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408e86cf-310a-497a-9038-0a2efbe718e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:08.337684 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408e86cf-310a-497a-9038-0a2efbe718e7 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:08.337962 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408e86cf-310a-497a-9038-0a2efbe718e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:08.338252 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408e86cf-310a-497a-9038-0a2efbe718e7 None admin] PATH_INFO: `/v3/projects/1f696f55becc4e879b6d1bac9d93d055` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:08.339410 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408e86cf-310a-497a-9038-0a2efbe718e7 None admin] RBAC: Authorizing `identity:get_project(project_id=1f696f55becc4e879b6d1bac9d93d055)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:08.340890 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408e86cf-310a-497a-9038-0a2efbe718e7 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:08.343203 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 44/93] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Sep 1 14:45:08 2022] GET /identity/v3/projects/1f696f55becc4e879b6d1bac9d93d055 => generated 298 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:45:08.343336 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408e86cf-310a-497a-9038-0a2efbe718e7 None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:08.384517 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34eefd00-4a8d-465f-bd1f-cd2ba6b4876d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:08.385508 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34eefd00-4a8d-465f-bd1f-cd2ba6b4876d None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:08.386012 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34eefd00-4a8d-465f-bd1f-cd2ba6b4876d None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:08.386395 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34eefd00-4a8d-465f-bd1f-cd2ba6b4876d None admin] PATH_INFO: `/v3/projects/13c8e0cc69454dbcadb4dce58c2565b0` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:08.387562 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34eefd00-4a8d-465f-bd1f-cd2ba6b4876d None admin] RBAC: Authorizing `identity:get_project(project_id=13c8e0cc69454dbcadb4dce58c2565b0)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:08.389029 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34eefd00-4a8d-465f-bd1f-cd2ba6b4876d None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:08.390988 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 50/94] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Sep 1 14:45:08 2022] GET /identity/v3/projects/13c8e0cc69454dbcadb4dce58c2565b0 => generated 312 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:45:08.391612 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34eefd00-4a8d-465f-bd1f-cd2ba6b4876d None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:08.455562 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:08.473687 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408e86cf-310a-497a-9038-0a2efbe718e7 None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:08.524859 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3535586-ab70-4608-b369-3d26fb8085ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:08.525816 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3535586-ab70-4608-b369-3d26fb8085ee None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:08.526192 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3535586-ab70-4608-b369-3d26fb8085ee None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:08.526588 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3535586-ab70-4608-b369-3d26fb8085ee None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:08.527419 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3535586-ab70-4608-b369-3d26fb8085ee None admin] RBAC: Adding query filter params (role.id=9996407d588d4a00836c8c50e7f37b1a, scope.project.id=1f696f55becc4e879b6d1bac9d93d055, user.id=4b65298713a24cada6ba75d786076e93) {{(pid=81341) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:45:08.527835 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3535586-ab70-4608-b369-3d26fb8085ee None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:08.529397 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3535586-ab70-4608-b369-3d26fb8085ee None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:08.535662 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 45/95] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Sep 1 14:45:08 2022] GET /identity/v3/role_assignments?user.id=4b65298713a24cada6ba75d786076e93&scope.project.id=1f696f55becc4e879b6d1bac9d93d055&role.id=9996407d588d4a00836c8c50e7f37b1a => generated 259 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:45:08.536639 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3535586-ab70-4608-b369-3d26fb8085ee None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:08.554515 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34eefd00-4a8d-465f-bd1f-cd2ba6b4876d None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:08.597259 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd923b3-d638-4816-bbad-e480f5e66c25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:08.598124 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd923b3-d638-4816-bbad-e480f5e66c25 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:08.598565 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd923b3-d638-4816-bbad-e480f5e66c25 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:08.599005 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd923b3-d638-4816-bbad-e480f5e66c25 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:08.599889 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd923b3-d638-4816-bbad-e480f5e66c25 None admin] RBAC: Adding query filter params (role.id=5d1feef0e13d4874928bd86258c4e3b2, scope.project.id=1f696f55becc4e879b6d1bac9d93d055, user.id=4b65298713a24cada6ba75d786076e93) {{(pid=81342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:45:08.600874 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd923b3-d638-4816-bbad-e480f5e66c25 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:08.602510 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd923b3-d638-4816-bbad-e480f5e66c25 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:08.609249 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 51/96] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Sep 1 14:45:08 2022] GET /identity/v3/role_assignments?user.id=4b65298713a24cada6ba75d786076e93&scope.project.id=1f696f55becc4e879b6d1bac9d93d055&role.id=5d1feef0e13d4874928bd86258c4e3b2 => generated 259 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:45:08.658625 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3535586-ab70-4608-b369-3d26fb8085ee None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:08.695740 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea706a59-dfbe-456b-b966-d08462bb080f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:08.696702 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea706a59-dfbe-456b-b966-d08462bb080f None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:08.696987 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea706a59-dfbe-456b-b966-d08462bb080f None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:08.697254 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea706a59-dfbe-456b-b966-d08462bb080f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:08.698027 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea706a59-dfbe-456b-b966-d08462bb080f None admin] RBAC: Adding query filter params (role.id=5d1feef0e13d4874928bd86258c4e3b2, scope.project.id=13c8e0cc69454dbcadb4dce58c2565b0, user.id=4b65298713a24cada6ba75d786076e93) {{(pid=81341) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:45:08.698372 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea706a59-dfbe-456b-b966-d08462bb080f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:08.699909 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea706a59-dfbe-456b-b966-d08462bb080f None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:08.709016 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 46/97] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Sep 1 14:45:08 2022] GET /identity/v3/role_assignments?user.id=4b65298713a24cada6ba75d786076e93&scope.project.id=13c8e0cc69454dbcadb4dce58c2565b0&role.id=5d1feef0e13d4874928bd86258c4e3b2 => generated 259 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:45:10.838719 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288d760a-c3b6-4de1-802d-7439ff7b0c7a None None] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:10.839225 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288d760a-c3b6-4de1-802d-7439ff7b0c7a None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:10.839298 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288d760a-c3b6-4de1-802d-7439ff7b0c7a None None] PATH_INFO: `/` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:10.840898 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 52/98] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 1 14:45:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:45:10.845895 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd445645-4243-4e87-b185-48bd8ad477de None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:10.846672 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd445645-4243-4e87-b185-48bd8ad477de None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:10.847087 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd445645-4243-4e87-b185-48bd8ad477de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:10.855985 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a643bb0-c861-4d16-b5fe-a9c81cad365a None None] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:10.856557 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a643bb0-c861-4d16-b5fe-a9c81cad365a None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:10.857071 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a643bb0-c861-4d16-b5fe-a9c81cad365a None None] PATH_INFO: `/` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:10.858934 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 53/99] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 1 14:45:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:45:10.864313 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beee09af-b7f0-447e-bfa5-a77fccbf1e18 None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:10.864826 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beee09af-b7f0-447e-bfa5-a77fccbf1e18 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:10.865322 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beee09af-b7f0-447e-bfa5-a77fccbf1e18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:10.871244 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-bd445645-4243-4e87-b185-48bd8ad477de None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:45:10.872506 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-bd445645-4243-4e87-b185-48bd8ad477de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:45:10.880285 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:10.887375 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-beee09af-b7f0-447e-bfa5-a77fccbf1e18 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:45:10.888896 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-beee09af-b7f0-447e-bfa5-a77fccbf1e18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:45:10.905556 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:10.967140 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 54/100] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 1 14:45:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 14:45:10.972299 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 47/101] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 1 14:45:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 14:45:11.093320 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1b071a-05ec-4c29-8bb6-67605c232d9e None None] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:11.093868 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1b071a-05ec-4c29-8bb6-67605c232d9e None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:11.094303 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1b071a-05ec-4c29-8bb6-67605c232d9e None None] PATH_INFO: `/` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:11.095811 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 55/102] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 1 14:45:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:45:11.101021 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf645cb-0217-4850-8869-9821fb453ad2 None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:11.101514 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf645cb-0217-4850-8869-9821fb453ad2 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:11.101960 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf645cb-0217-4850-8869-9821fb453ad2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:11.120346 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-acf645cb-0217-4850-8869-9821fb453ad2 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:45:11.121805 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-acf645cb-0217-4850-8869-9821fb453ad2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:45:11.189270 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 48/103] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 1 14:45:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 14:45:11.263402 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1b071a-05ec-4c29-8bb6-67605c232d9e None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:11.372880 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1b071a-05ec-4c29-8bb6-67605c232d9e None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:11.409337 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90305e4c-af36-4694-a614-b1337e5fcc03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:11.409976 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90305e4c-af36-4694-a614-b1337e5fcc03 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:11.410278 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90305e4c-af36-4694-a614-b1337e5fcc03 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:11.410563 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90305e4c-af36-4694-a614-b1337e5fcc03 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:11.421615 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90305e4c-af36-4694-a614-b1337e5fcc03 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 14:45:11.426662 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application [None req-90305e4c-af36-4694-a614-b1337e5fcc03 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 14:45:11.426662 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:45:11.426662 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:45:11.426662 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:45:11.426662 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:45:11.426662 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:45:11.426662 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:45:11.426662 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:45:11.426662 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:45:11.426662 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:45:11.426662 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:45:11.426662 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:45:11.426662 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 14:45:11.426662 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 14:45:11.426662 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 14:45:11.426662 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 14:45:11.427533 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 14:45:11.427533 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 14:45:11.427533 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 14:45:11.427533 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 14:45:11.427533 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:45:11.427533 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:45:11.427533 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:45:11.427533 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:45:11.427533 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:45:11.427533 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:45:11.427533 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:45:11.427533 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application with Lock( Sep 01 14:45:11.427533 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:45:11.427533 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:45:11.427533 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:45:11.427533 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:45:11.428859 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:45:11.428859 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:45:11.428859 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:45:11.428859 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:45:11.428859 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 14:45:11.428859 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 14:45:11.428859 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 14:45:11.428859 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 14:45:11.428859 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 14:45:11.428859 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 14:45:11.428859 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 14:45:11.428859 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Sep 01 14:45:11.428859 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 56/104] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Sep 1 14:45:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:45:11.434312 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf645cb-0217-4850-8869-9821fb453ad2 None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:11.474730 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90305e4c-af36-4694-a614-b1337e5fcc03 None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:11.488154 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:11.598070 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90305e4c-af36-4694-a614-b1337e5fcc03 None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:11.600730 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:11.631387 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf645cb-0217-4850-8869-9821fb453ad2 None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:11.639198 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032272c4-8ba0-417e-af6b-3a4c1ce5a624 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:11.639682 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032272c4-8ba0-417e-af6b-3a4c1ce5a624 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:11.639972 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032272c4-8ba0-417e-af6b-3a4c1ce5a624 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:11.640254 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032272c4-8ba0-417e-af6b-3a4c1ce5a624 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:11.651825 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032272c4-8ba0-417e-af6b-3a4c1ce5a624 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 14:45:11.657128 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application [None req-032272c4-8ba0-417e-af6b-3a4c1ce5a624 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 14:45:11.657128 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:45:11.657128 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:45:11.657128 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:45:11.657128 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:45:11.657128 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:45:11.657128 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:45:11.657128 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:45:11.657128 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:45:11.657128 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:45:11.657128 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:45:11.657128 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:45:11.657128 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 14:45:11.657128 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 14:45:11.657128 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 14:45:11.657128 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 14:45:11.657996 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 14:45:11.657996 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 14:45:11.657996 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 14:45:11.657996 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 14:45:11.657996 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:45:11.657996 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:45:11.657996 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:45:11.657996 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:45:11.657996 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:45:11.657996 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:45:11.657996 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:45:11.657996 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application with Lock( Sep 01 14:45:11.657996 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:45:11.657996 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:45:11.657996 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:45:11.657996 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:45:11.658817 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:45:11.658817 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:45:11.658817 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:45:11.658817 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:45:11.658817 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 14:45:11.658817 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 14:45:11.658817 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 14:45:11.658817 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 14:45:11.658817 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 14:45:11.658817 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 14:45:11.658817 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 14:45:11.658817 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Sep 01 14:45:11.658817 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 57/105] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Sep 1 14:45:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:45:11.659936 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6df194-a79d-4a62-96af-039cf8eb1605 None None] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:11.660087 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6df194-a79d-4a62-96af-039cf8eb1605 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:11.660367 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6df194-a79d-4a62-96af-039cf8eb1605 None None] PATH_INFO: `/` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:11.661854 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 58/106] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 1 14:45:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:45:11.666880 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6df194-a79d-4a62-96af-039cf8eb1605 None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:11.673932 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c9f02c-b791-41dc-92bc-33d849c61310 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:11.674843 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c9f02c-b791-41dc-92bc-33d849c61310 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:11.675335 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c9f02c-b791-41dc-92bc-33d849c61310 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:11.675826 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c9f02c-b791-41dc-92bc-33d849c61310 None admin] PATH_INFO: `/v3/roles` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:11.676684 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c9f02c-b791-41dc-92bc-33d849c61310 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81341) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:45:11.677230 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c9f02c-b791-41dc-92bc-33d849c61310 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:11.679461 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c9f02c-b791-41dc-92bc-33d849c61310 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:11.688226 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 49/107] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Sep 1 14:45:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:45:11.689328 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c9f02c-b791-41dc-92bc-33d849c61310 None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:11.745284 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:11.781216 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6df194-a79d-4a62-96af-039cf8eb1605 None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:11.797273 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c9f02c-b791-41dc-92bc-33d849c61310 None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:11.817879 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21a4d18-8c03-43e6-bcc8-f083f91ca17a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:11.818531 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21a4d18-8c03-43e6-bcc8-f083f91ca17a None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:11.819146 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21a4d18-8c03-43e6-bcc8-f083f91ca17a None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:11.819146 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21a4d18-8c03-43e6-bcc8-f083f91ca17a None admin] PATH_INFO: `/v3/roles` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:11.819779 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21a4d18-8c03-43e6-bcc8-f083f91ca17a None admin] RBAC: Adding query filter params (name=member) {{(pid=81342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:45:11.820093 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21a4d18-8c03-43e6-bcc8-f083f91ca17a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:11.821758 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21a4d18-8c03-43e6-bcc8-f083f91ca17a None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:11.829969 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 59/108] 10.222.0.33 () {56 vars in 1101 bytes} [Thu Sep 1 14:45:11 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:45:11.831148 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c0055a-a244-4184-a2a3-831d11d69169 None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:11.831439 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c0055a-a244-4184-a2a3-831d11d69169 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:11.831716 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c0055a-a244-4184-a2a3-831d11d69169 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:11.838700 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84911f19-81a4-4603-b661-6eb4021cb7f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:11.839553 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84911f19-81a4-4603-b661-6eb4021cb7f8 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:11.839841 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84911f19-81a4-4603-b661-6eb4021cb7f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:11.840117 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84911f19-81a4-4603-b661-6eb4021cb7f8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:11.841236 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84911f19-81a4-4603-b661-6eb4021cb7f8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:11.842821 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84911f19-81a4-4603-b661-6eb4021cb7f8 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:11.844399 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 50/109] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 1 14:45:11 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:45:11.850049 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-46c0055a-a244-4184-a2a3-831d11d69169 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:45:11.850955 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f94300a-42a5-4fbe-9b8e-0972e4480f01 None None] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:11.851028 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f94300a-42a5-4fbe-9b8e-0972e4480f01 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:11.851268 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f94300a-42a5-4fbe-9b8e-0972e4480f01 None None] PATH_INFO: `/` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:11.851767 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-46c0055a-a244-4184-a2a3-831d11d69169 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:45:11.852664 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 51/110] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 1 14:45:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:45:11.855953 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f94300a-42a5-4fbe-9b8e-0972e4480f01 None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:11.859397 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:11.921819 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 60/111] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 1 14:45:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 14:45:11.922852 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c0055a-a244-4184-a2a3-831d11d69169 None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:11.982609 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f94300a-42a5-4fbe-9b8e-0972e4480f01 None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:11.986698 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:12.017663 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b4ed7b-9165-4cc8-95ee-e27fe01cc9c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:12.018329 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b4ed7b-9165-4cc8-95ee-e27fe01cc9c1 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:12.018623 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b4ed7b-9165-4cc8-95ee-e27fe01cc9c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:12.018889 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b4ed7b-9165-4cc8-95ee-e27fe01cc9c1 None admin] PATH_INFO: `/v3/users/abbabc07754b486599ec507190d24c97` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:12.020024 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b4ed7b-9165-4cc8-95ee-e27fe01cc9c1 None admin] RBAC: Authorizing `identity:get_user(user_id=abbabc07754b486599ec507190d24c97)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:12.021616 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b4ed7b-9165-4cc8-95ee-e27fe01cc9c1 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:12.023899 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 52/112] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Sep 1 14:45:11 2022] GET /identity/v3/users/abbabc07754b486599ec507190d24c97 => generated 247 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:45:12.024522 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b4ed7b-9165-4cc8-95ee-e27fe01cc9c1 None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:12.040170 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c0055a-a244-4184-a2a3-831d11d69169 None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:12.073402 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50925281-dae9-4901-9268-f5c40b17229b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:12.074282 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50925281-dae9-4901-9268-f5c40b17229b None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:12.074744 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50925281-dae9-4901-9268-f5c40b17229b None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:12.075017 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50925281-dae9-4901-9268-f5c40b17229b None admin] PATH_INFO: `/v3/users/4b65298713a24cada6ba75d786076e93` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:12.076133 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50925281-dae9-4901-9268-f5c40b17229b None admin] RBAC: Authorizing `identity:get_user(user_id=4b65298713a24cada6ba75d786076e93)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:12.077573 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50925281-dae9-4901-9268-f5c40b17229b None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:12.079340 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 61/113] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Sep 1 14:45:11 2022] GET /identity/v3/users/4b65298713a24cada6ba75d786076e93 => generated 275 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:45:12.085202 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019f01c6-3d51-46f0-9fb0-268b67c69e52 None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:12.085487 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019f01c6-3d51-46f0-9fb0-268b67c69e52 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:12.085764 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019f01c6-3d51-46f0-9fb0-268b67c69e52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:12.097037 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:12.104199 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-019f01c6-3d51-46f0-9fb0-268b67c69e52 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:45:12.105326 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-019f01c6-3d51-46f0-9fb0-268b67c69e52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:45:12.131007 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b4ed7b-9165-4cc8-95ee-e27fe01cc9c1 None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:12.166024 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcbe698-f627-41b3-91fc-6bc058e486af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:12.166656 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcbe698-f627-41b3-91fc-6bc058e486af None admin] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:12.166939 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcbe698-f627-41b3-91fc-6bc058e486af None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:12.167209 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcbe698-f627-41b3-91fc-6bc058e486af None admin] PATH_INFO: `/v3/projects` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:12.167859 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcbe698-f627-41b3-91fc-6bc058e486af None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:12.169311 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcbe698-f627-41b3-91fc-6bc058e486af None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:12.182387 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 62/114] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 1 14:45:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 14:45:12.182387 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fd7195-e51e-40f9-9798-96579372cf94 None None] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:12.182387 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fd7195-e51e-40f9-9798-96579372cf94 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:12.182387 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fd7195-e51e-40f9-9798-96579372cf94 None None] PATH_INFO: `/` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:12.182387 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 63/115] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 1 14:45:12 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:45:12.182387 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fd7195-e51e-40f9-9798-96579372cf94 None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:12.209058 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 53/116] 10.222.0.33 () {60 vars in 1141 bytes} [Thu Sep 1 14:45:12 2022] POST /identity/v3/projects => generated 302 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:45:12.210466 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcbe698-f627-41b3-91fc-6bc058e486af None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:12.266180 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:12.330561 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fd7195-e51e-40f9-9798-96579372cf94 None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:12.332588 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcbe698-f627-41b3-91fc-6bc058e486af None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:12.365612 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:12.376345 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9986ef3-f5fe-4628-9751-1c2d8bd6ee50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:12.377001 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9986ef3-f5fe-4628-9751-1c2d8bd6ee50 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:12.377284 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9986ef3-f5fe-4628-9751-1c2d8bd6ee50 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:12.377554 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9986ef3-f5fe-4628-9751-1c2d8bd6ee50 None admin] PATH_INFO: `/v3/projects/1f696f55becc4e879b6d1bac9d93d055` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:12.378728 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9986ef3-f5fe-4628-9751-1c2d8bd6ee50 None admin] RBAC: Authorizing `identity:get_project(project_id=1f696f55becc4e879b6d1bac9d93d055)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:12.380187 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9986ef3-f5fe-4628-9751-1c2d8bd6ee50 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:12.382837 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 64/117] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Sep 1 14:45:12 2022] GET /identity/v3/projects/1f696f55becc4e879b6d1bac9d93d055 => generated 298 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:45:12.383961 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b86b58-ac84-4f43-a4d9-9d5fe4955ec9 None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:12.384240 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b86b58-ac84-4f43-a4d9-9d5fe4955ec9 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:12.384518 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b86b58-ac84-4f43-a4d9-9d5fe4955ec9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:12.390267 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fca59f-d194-4246-b076-e0a4337b9bdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:12.391338 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fca59f-d194-4246-b076-e0a4337b9bdc None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:12.392438 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fca59f-d194-4246-b076-e0a4337b9bdc None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:12.392788 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fca59f-d194-4246-b076-e0a4337b9bdc None admin] PATH_INFO: `/v3/projects/13c8e0cc69454dbcadb4dce58c2565b0` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:12.393959 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fca59f-d194-4246-b076-e0a4337b9bdc None admin] RBAC: Authorizing `identity:get_project(project_id=13c8e0cc69454dbcadb4dce58c2565b0)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:12.395970 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fca59f-d194-4246-b076-e0a4337b9bdc None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:12.399845 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 54/118] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Sep 1 14:45:12 2022] GET /identity/v3/projects/13c8e0cc69454dbcadb4dce58c2565b0 => generated 312 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:45:12.400277 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fca59f-d194-4246-b076-e0a4337b9bdc None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:12.410702 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-64b86b58-ac84-4f43-a4d9-9d5fe4955ec9 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:45:12.411904 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-64b86b58-ac84-4f43-a4d9-9d5fe4955ec9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:45:12.493757 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 65/119] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 1 14:45:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 14:45:12.494926 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b86b58-ac84-4f43-a4d9-9d5fe4955ec9 None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:12.515389 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fca59f-d194-4246-b076-e0a4337b9bdc None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:12.554499 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e570e74-e783-4eb7-a574-a4681b6d8b34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:12.555237 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e570e74-e783-4eb7-a574-a4681b6d8b34 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:12.555583 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e570e74-e783-4eb7-a574-a4681b6d8b34 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:12.555927 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e570e74-e783-4eb7-a574-a4681b6d8b34 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:12.566841 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e570e74-e783-4eb7-a574-a4681b6d8b34 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 14:45:12.572691 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application [None req-0e570e74-e783-4eb7-a574-a4681b6d8b34 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 14:45:12.572691 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:45:12.572691 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:45:12.572691 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:45:12.572691 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:45:12.572691 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:45:12.572691 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:45:12.572691 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:45:12.572691 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:45:12.572691 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:45:12.572691 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:45:12.572691 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:45:12.572691 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 14:45:12.572691 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 14:45:12.572691 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 14:45:12.572691 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 14:45:12.573610 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 14:45:12.573610 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 14:45:12.573610 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 14:45:12.573610 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 14:45:12.573610 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:45:12.573610 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:45:12.573610 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:45:12.573610 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:45:12.573610 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:45:12.573610 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:45:12.573610 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:45:12.573610 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application with Lock( Sep 01 14:45:12.573610 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:45:12.573610 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:45:12.573610 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:45:12.573610 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:45:12.574437 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:45:12.574437 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:45:12.574437 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:45:12.574437 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:45:12.574437 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 14:45:12.574437 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 14:45:12.574437 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 14:45:12.574437 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 14:45:12.574437 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 14:45:12.574437 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 14:45:12.574437 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 14:45:12.574437 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Sep 01 14:45:12.574437 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 55/120] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Sep 1 14:45:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:45:12.575096 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e570e74-e783-4eb7-a574-a4681b6d8b34 None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:12.579747 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:12.628383 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:12.630099 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b86b58-ac84-4f43-a4d9-9d5fe4955ec9 None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:12.678484 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341ccb3a-9d62-4612-8eac-b25d74aba874 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:12.679622 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341ccb3a-9d62-4612-8eac-b25d74aba874 None admin] REQUEST_METHOD: `PUT` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:12.680437 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341ccb3a-9d62-4612-8eac-b25d74aba874 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:12.680890 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341ccb3a-9d62-4612-8eac-b25d74aba874 None admin] PATH_INFO: `/v3/projects/1f696f55becc4e879b6d1bac9d93d055/users/abbabc07754b486599ec507190d24c97/roles/2a2adb769e55476b82e5014897e0a9c0` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:12.683881 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341ccb3a-9d62-4612-8eac-b25d74aba874 None admin] RBAC: Authorizing `identity:create_grant(project_id=1f696f55becc4e879b6d1bac9d93d055, user_id=abbabc07754b486599ec507190d24c97, role_id=2a2adb769e55476b82e5014897e0a9c0)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:12.686034 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341ccb3a-9d62-4612-8eac-b25d74aba874 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:12.689326 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e570e74-e783-4eb7-a574-a4681b6d8b34 None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:12.706511 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 66/121] 10.222.0.33 () {58 vars in 1439 bytes} [Thu Sep 1 14:45:12 2022] PUT /identity/v3/projects/1f696f55becc4e879b6d1bac9d93d055/users/abbabc07754b486599ec507190d24c97/roles/2a2adb769e55476b82e5014897e0a9c0 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:45:12.707692 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341ccb3a-9d62-4612-8eac-b25d74aba874 None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:12.749570 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e324f42a-bd0e-4e07-bbbe-3099dd0196e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:12.750552 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e324f42a-bd0e-4e07-bbbe-3099dd0196e8 None admin] REQUEST_METHOD: `PUT` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:12.751004 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e324f42a-bd0e-4e07-bbbe-3099dd0196e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:12.751446 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e324f42a-bd0e-4e07-bbbe-3099dd0196e8 None admin] PATH_INFO: `/v3/projects/13c8e0cc69454dbcadb4dce58c2565b0/users/4b65298713a24cada6ba75d786076e93/roles/5d1feef0e13d4874928bd86258c4e3b2` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:12.758502 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e324f42a-bd0e-4e07-bbbe-3099dd0196e8 None admin] RBAC: Authorizing `identity:create_grant(project_id=13c8e0cc69454dbcadb4dce58c2565b0, user_id=4b65298713a24cada6ba75d786076e93, role_id=5d1feef0e13d4874928bd86258c4e3b2)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:12.760193 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e324f42a-bd0e-4e07-bbbe-3099dd0196e8 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:12.788785 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 56/122] 10.222.0.33 () {58 vars in 1439 bytes} [Thu Sep 1 14:45:12 2022] PUT /identity/v3/projects/13c8e0cc69454dbcadb4dce58c2565b0/users/4b65298713a24cada6ba75d786076e93/roles/5d1feef0e13d4874928bd86258c4e3b2 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:45:12.789842 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e324f42a-bd0e-4e07-bbbe-3099dd0196e8 None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:12.867181 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341ccb3a-9d62-4612-8eac-b25d74aba874 None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:12.900791 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:12.922206 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e324f42a-bd0e-4e07-bbbe-3099dd0196e8 None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:12.935540 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d94174-fc1a-429d-84df-fc5cce956127 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:12.936638 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d94174-fc1a-429d-84df-fc5cce956127 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:12.936929 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d94174-fc1a-429d-84df-fc5cce956127 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:12.937256 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d94174-fc1a-429d-84df-fc5cce956127 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:12.938643 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:12.954082 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d94174-fc1a-429d-84df-fc5cce956127 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 14:45:12.961387 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application [None req-a0d94174-fc1a-429d-84df-fc5cce956127 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 14:45:12.961387 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:45:12.961387 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:45:12.961387 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:45:12.961387 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:45:12.961387 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:45:12.961387 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:45:12.961387 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:45:12.961387 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:45:12.961387 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:45:12.961387 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:45:12.961387 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:45:12.961387 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 14:45:12.961387 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 14:45:12.961387 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 14:45:12.961387 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 14:45:12.962381 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 14:45:12.962381 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 14:45:12.962381 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 14:45:12.962381 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 14:45:12.962381 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:45:12.962381 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:45:12.962381 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:45:12.962381 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:45:12.962381 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:45:12.962381 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:45:12.962381 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:45:12.962381 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application with Lock( Sep 01 14:45:12.962381 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:45:12.962381 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:45:12.962381 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:45:12.962381 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:45:12.963669 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:45:12.963669 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:45:12.963669 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:45:12.963669 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:45:12.963669 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 14:45:12.963669 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 14:45:12.963669 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 14:45:12.963669 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 14:45:12.963669 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 14:45:12.963669 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 14:45:12.963669 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 14:45:12.963669 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Sep 01 14:45:12.963669 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 67/123] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Sep 1 14:45:12 2022] GET /identity/v3/roles/reader => generated 84 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:45:12.963669 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d94174-fc1a-429d-84df-fc5cce956127 None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:12.978929 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81be8b6-4ffb-4e04-934c-ac12e179e3af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:12.980460 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81be8b6-4ffb-4e04-934c-ac12e179e3af None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:12.980999 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81be8b6-4ffb-4e04-934c-ac12e179e3af None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:12.981446 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81be8b6-4ffb-4e04-934c-ac12e179e3af None admin] PATH_INFO: `/v3/roles` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:12.982674 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81be8b6-4ffb-4e04-934c-ac12e179e3af None admin] RBAC: Adding query filter params (name=member) {{(pid=81341) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:45:12.983176 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81be8b6-4ffb-4e04-934c-ac12e179e3af None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:12.985618 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81be8b6-4ffb-4e04-934c-ac12e179e3af None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:12.998153 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 57/124] 10.222.0.33 () {56 vars in 1101 bytes} [Thu Sep 1 14:45:12 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:45:12.999791 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81be8b6-4ffb-4e04-934c-ac12e179e3af None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:13.086918 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d94174-fc1a-429d-84df-fc5cce956127 None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:13.110283 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81be8b6-4ffb-4e04-934c-ac12e179e3af None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:13.125844 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11354a6c-8e70-494e-b1ba-e81739da7ccd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:13.126785 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11354a6c-8e70-494e-b1ba-e81739da7ccd None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:13.127276 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11354a6c-8e70-494e-b1ba-e81739da7ccd None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:13.127733 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11354a6c-8e70-494e-b1ba-e81739da7ccd None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:13.140070 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11354a6c-8e70-494e-b1ba-e81739da7ccd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 14:45:13.145395 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application [None req-11354a6c-8e70-494e-b1ba-e81739da7ccd None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 14:45:13.145395 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:45:13.145395 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:45:13.145395 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:45:13.145395 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:45:13.145395 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:45:13.145395 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:45:13.145395 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:45:13.145395 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:45:13.145395 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:45:13.145395 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:45:13.145395 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:45:13.145395 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 14:45:13.145395 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 14:45:13.145395 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 14:45:13.145395 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 14:45:13.146317 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 14:45:13.146317 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 14:45:13.146317 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 14:45:13.146317 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 14:45:13.146317 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:45:13.146317 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:45:13.146317 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:45:13.146317 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:45:13.146317 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:45:13.146317 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:45:13.146317 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:45:13.146317 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application with Lock( Sep 01 14:45:13.146317 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:45:13.146317 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:45:13.146317 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:45:13.146317 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:45:13.147223 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:45:13.147223 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:45:13.147223 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:45:13.147223 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:45:13.147223 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 14:45:13.147223 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 14:45:13.147223 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 14:45:13.147223 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 14:45:13.147223 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 14:45:13.147223 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 14:45:13.147223 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 14:45:13.147223 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Sep 01 14:45:13.147837 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2e531b-a756-4566-8b55-722390caaad0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:13.147837 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2e531b-a756-4566-8b55-722390caaad0 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:13.147837 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2e531b-a756-4566-8b55-722390caaad0 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:13.147837 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2e531b-a756-4566-8b55-722390caaad0 None admin] PATH_INFO: `/v3/roles` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:13.148089 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2e531b-a756-4566-8b55-722390caaad0 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81341) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:45:13.148954 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2e531b-a756-4566-8b55-722390caaad0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:13.150113 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 68/125] 10.222.0.33 () {56 vars in 1114 bytes} [Thu Sep 1 14:45:12 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:45:13.154373 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2e531b-a756-4566-8b55-722390caaad0 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:13.155460 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11354a6c-8e70-494e-b1ba-e81739da7ccd None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:13.161075 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 58/126] 10.222.0.33 () {56 vars in 1101 bytes} [Thu Sep 1 14:45:12 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:45:13.167435 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2e531b-a756-4566-8b55-722390caaad0 None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:13.200376 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:13.227182 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:13.329293 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11354a6c-8e70-494e-b1ba-e81739da7ccd None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:13.358922 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2e531b-a756-4566-8b55-722390caaad0 None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:13.394218 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79314a8-e4f9-4c9b-befc-9589a9739427 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:13.394928 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79314a8-e4f9-4c9b-befc-9589a9739427 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:13.395213 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79314a8-e4f9-4c9b-befc-9589a9739427 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:13.395491 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79314a8-e4f9-4c9b-befc-9589a9739427 None admin] PATH_INFO: `/v3/users/0159fb4a6f8a45249190b6bae1b464c1` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:13.402511 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3286599c-e11c-4a29-a428-bbb3554c20cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:13.403181 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3286599c-e11c-4a29-a428-bbb3554c20cf None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:13.403470 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3286599c-e11c-4a29-a428-bbb3554c20cf None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:13.403743 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3286599c-e11c-4a29-a428-bbb3554c20cf None admin] PATH_INFO: `/v3/users/4b65298713a24cada6ba75d786076e93` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:13.411529 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79314a8-e4f9-4c9b-befc-9589a9739427 None admin] RBAC: Authorizing `identity:get_user(user_id=0159fb4a6f8a45249190b6bae1b464c1)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:13.411685 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3286599c-e11c-4a29-a428-bbb3554c20cf None admin] RBAC: Authorizing `identity:get_user(user_id=4b65298713a24cada6ba75d786076e93)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:13.413516 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79314a8-e4f9-4c9b-befc-9589a9739427 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:13.413693 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3286599c-e11c-4a29-a428-bbb3554c20cf None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:13.415633 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 59/127] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Sep 1 14:45:13 2022] GET /identity/v3/users/0159fb4a6f8a45249190b6bae1b464c1 => generated 289 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 14:45:13.415760 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 69/128] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Sep 1 14:45:13 2022] GET /identity/v3/users/4b65298713a24cada6ba75d786076e93 => generated 275 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:45:13.416243 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79314a8-e4f9-4c9b-befc-9589a9739427 None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:13.422635 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3286599c-e11c-4a29-a428-bbb3554c20cf None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:13.518776 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:13.537654 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79314a8-e4f9-4c9b-befc-9589a9739427 None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:13.602884 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a92b9a6-50dd-4869-b2f2-487047ef95ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:13.603500 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a92b9a6-50dd-4869-b2f2-487047ef95ce None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:13.603783 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a92b9a6-50dd-4869-b2f2-487047ef95ce None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:13.604055 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a92b9a6-50dd-4869-b2f2-487047ef95ce None admin] PATH_INFO: `/v3/roles` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:13.604728 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a92b9a6-50dd-4869-b2f2-487047ef95ce None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81341) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:45:13.605033 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a92b9a6-50dd-4869-b2f2-487047ef95ce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:13.606929 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3286599c-e11c-4a29-a428-bbb3554c20cf None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:13.609708 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a92b9a6-50dd-4869-b2f2-487047ef95ce None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:13.619630 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:13.626289 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 60/129] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Sep 1 14:45:13 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:45:13.627168 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a92b9a6-50dd-4869-b2f2-487047ef95ce None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:13.675919 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7df6d7d-3095-48f6-8052-10e383475c90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:13.676540 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7df6d7d-3095-48f6-8052-10e383475c90 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:13.676823 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7df6d7d-3095-48f6-8052-10e383475c90 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:13.677099 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7df6d7d-3095-48f6-8052-10e383475c90 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:13.794723 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a92b9a6-50dd-4869-b2f2-487047ef95ce None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:13.837820 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7df6d7d-3095-48f6-8052-10e383475c90 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:13.839342 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7df6d7d-3095-48f6-8052-10e383475c90 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:13.858023 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419ff9e0-4fe7-4c45-a13c-43d989dfdd08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:13.858697 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419ff9e0-4fe7-4c45-a13c-43d989dfdd08 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:13.858981 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419ff9e0-4fe7-4c45-a13c-43d989dfdd08 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:13.859258 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419ff9e0-4fe7-4c45-a13c-43d989dfdd08 None admin] PATH_INFO: `/v3/projects/1f696f55becc4e879b6d1bac9d93d055` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:13.861747 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419ff9e0-4fe7-4c45-a13c-43d989dfdd08 None admin] RBAC: Authorizing `identity:get_project(project_id=1f696f55becc4e879b6d1bac9d93d055)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:13.863243 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419ff9e0-4fe7-4c45-a13c-43d989dfdd08 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:13.867706 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 61/130] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Sep 1 14:45:13 2022] GET /identity/v3/projects/1f696f55becc4e879b6d1bac9d93d055 => generated 298 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:45:13.868519 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419ff9e0-4fe7-4c45-a13c-43d989dfdd08 None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:13.934741 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:13.979898 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419ff9e0-4fe7-4c45-a13c-43d989dfdd08 None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:14.016636 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f6ab82-7dc7-4cd9-b4c0-7b31ddd4c078 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:14.017338 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f6ab82-7dc7-4cd9-b4c0-7b31ddd4c078 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:14.017630 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f6ab82-7dc7-4cd9-b4c0-7b31ddd4c078 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:14.017904 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f6ab82-7dc7-4cd9-b4c0-7b31ddd4c078 None admin] PATH_INFO: `/v3/users/4b65298713a24cada6ba75d786076e93` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:14.019103 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f6ab82-7dc7-4cd9-b4c0-7b31ddd4c078 None admin] RBAC: Authorizing `identity:get_user(user_id=4b65298713a24cada6ba75d786076e93)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:14.020572 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f6ab82-7dc7-4cd9-b4c0-7b31ddd4c078 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:14.022897 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 62/131] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Sep 1 14:45:13 2022] GET /identity/v3/users/4b65298713a24cada6ba75d786076e93 => generated 275 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:45:14.023471 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f6ab82-7dc7-4cd9-b4c0-7b31ddd4c078 None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:14.063311 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 70/132] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Sep 1 14:45:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 641 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 01 14:45:14.064273 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7df6d7d-3095-48f6-8052-10e383475c90 None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:14.067706 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:14.149900 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:14.173246 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f6ab82-7dc7-4cd9-b4c0-7b31ddd4c078 None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:14.179808 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7df6d7d-3095-48f6-8052-10e383475c90 None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:14.217248 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddb7748-1432-4fa0-8637-af49ed64efb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:14.217853 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddb7748-1432-4fa0-8637-af49ed64efb6 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:14.218143 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddb7748-1432-4fa0-8637-af49ed64efb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:14.218450 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddb7748-1432-4fa0-8637-af49ed64efb6 None admin] PATH_INFO: `/v3/projects/1f696f55becc4e879b6d1bac9d93d055` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:14.219894 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ddb7748-1432-4fa0-8637-af49ed64efb6 None admin] RBAC: Authorizing `identity:get_project(project_id=1f696f55becc4e879b6d1bac9d93d055)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:14.221488 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ddb7748-1432-4fa0-8637-af49ed64efb6 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:14.223190 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 71/133] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Sep 1 14:45:14 2022] GET /identity/v3/projects/1f696f55becc4e879b6d1bac9d93d055 => generated 298 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:45:14.224052 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddb7748-1432-4fa0-8637-af49ed64efb6 None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:14.233296 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6ee6db-2673-4928-bb12-3803fdd33811 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:14.233975 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6ee6db-2673-4928-bb12-3803fdd33811 None admin] REQUEST_METHOD: `PUT` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:14.234275 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6ee6db-2673-4928-bb12-3803fdd33811 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:14.234559 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6ee6db-2673-4928-bb12-3803fdd33811 None admin] PATH_INFO: `/v3/projects/1f696f55becc4e879b6d1bac9d93d055/users/4b65298713a24cada6ba75d786076e93/roles/5d1feef0e13d4874928bd86258c4e3b2` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:14.239524 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6ee6db-2673-4928-bb12-3803fdd33811 None admin] RBAC: Authorizing `identity:create_grant(project_id=1f696f55becc4e879b6d1bac9d93d055, user_id=4b65298713a24cada6ba75d786076e93, role_id=5d1feef0e13d4874928bd86258c4e3b2)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:14.241568 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6ee6db-2673-4928-bb12-3803fdd33811 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:14.272839 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 63/134] 10.222.0.33 () {58 vars in 1439 bytes} [Thu Sep 1 14:45:14 2022] PUT /identity/v3/projects/1f696f55becc4e879b6d1bac9d93d055/users/4b65298713a24cada6ba75d786076e93/roles/5d1feef0e13d4874928bd86258c4e3b2 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:45:14.273439 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6ee6db-2673-4928-bb12-3803fdd33811 None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:14.306676 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:14.344960 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddb7748-1432-4fa0-8637-af49ed64efb6 None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:14.384282 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b94fa6a-5da2-4078-96e3-4f437255213d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:14.384926 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b94fa6a-5da2-4078-96e3-4f437255213d None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:14.385256 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b94fa6a-5da2-4078-96e3-4f437255213d None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:14.385539 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b94fa6a-5da2-4078-96e3-4f437255213d None admin] PATH_INFO: `/v3/projects/1f696f55becc4e879b6d1bac9d93d055` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:14.391183 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b94fa6a-5da2-4078-96e3-4f437255213d None admin] RBAC: Authorizing `identity:get_project(project_id=1f696f55becc4e879b6d1bac9d93d055)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:14.392753 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b94fa6a-5da2-4078-96e3-4f437255213d None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:14.394666 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 72/135] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Sep 1 14:45:14 2022] GET /identity/v3/projects/1f696f55becc4e879b6d1bac9d93d055 => generated 298 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:45:14.400945 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b94fa6a-5da2-4078-96e3-4f437255213d None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:14.470754 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6ee6db-2673-4928-bb12-3803fdd33811 None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:14.507374 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024aba95-e770-4467-89a8-569ac3582623 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:14.508119 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024aba95-e770-4467-89a8-569ac3582623 None admin] REQUEST_METHOD: `PUT` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:14.508411 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024aba95-e770-4467-89a8-569ac3582623 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:14.508686 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024aba95-e770-4467-89a8-569ac3582623 None admin] PATH_INFO: `/v3/projects/1f696f55becc4e879b6d1bac9d93d055/users/4b65298713a24cada6ba75d786076e93/roles/9996407d588d4a00836c8c50e7f37b1a` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:14.511194 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024aba95-e770-4467-89a8-569ac3582623 None admin] RBAC: Authorizing `identity:create_grant(project_id=1f696f55becc4e879b6d1bac9d93d055, user_id=4b65298713a24cada6ba75d786076e93, role_id=9996407d588d4a00836c8c50e7f37b1a)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:14.512854 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024aba95-e770-4467-89a8-569ac3582623 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:14.512977 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b94fa6a-5da2-4078-96e3-4f437255213d None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:14.529210 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 64/136] 10.222.0.33 () {58 vars in 1439 bytes} [Thu Sep 1 14:45:14 2022] PUT /identity/v3/projects/1f696f55becc4e879b6d1bac9d93d055/users/4b65298713a24cada6ba75d786076e93/roles/9996407d588d4a00836c8c50e7f37b1a => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:45:14.530553 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277fd39e-aa3e-473b-8f8c-e833d8cdd9c4 None None] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:14.530848 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277fd39e-aa3e-473b-8f8c-e833d8cdd9c4 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:14.531129 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277fd39e-aa3e-473b-8f8c-e833d8cdd9c4 None None] PATH_INFO: `/` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:14.533038 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 65/137] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 1 14:45:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:45:14.539262 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ffa4a3-b4f1-4d29-a3b2-2d002c7d05de None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:14.539674 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ffa4a3-b4f1-4d29-a3b2-2d002c7d05de None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:14.539958 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ffa4a3-b4f1-4d29-a3b2-2d002c7d05de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:14.553881 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3801d6-0f54-4d88-b758-8e4f96723658 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:14.554634 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3801d6-0f54-4d88-b758-8e4f96723658 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:14.554918 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3801d6-0f54-4d88-b758-8e4f96723658 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:14.555192 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3801d6-0f54-4d88-b758-8e4f96723658 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:14.555959 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3801d6-0f54-4d88-b758-8e4f96723658 None admin] RBAC: Adding query filter params (role.id=869a5d710fb347a195e8f7b2cd3f0750, scope.project.id=1f696f55becc4e879b6d1bac9d93d055, user.id=0159fb4a6f8a45249190b6bae1b464c1) {{(pid=81342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:45:14.556272 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3801d6-0f54-4d88-b758-8e4f96723658 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:14.557797 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3801d6-0f54-4d88-b758-8e4f96723658 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:14.567470 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-50ffa4a3-b4f1-4d29-a3b2-2d002c7d05de None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:45:14.568719 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-50ffa4a3-b4f1-4d29-a3b2-2d002c7d05de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:45:14.573257 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 73/138] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Sep 1 14:45:14 2022] GET /identity/v3/role_assignments?user.id=0159fb4a6f8a45249190b6bae1b464c1&scope.project.id=1f696f55becc4e879b6d1bac9d93d055&role.id=869a5d710fb347a195e8f7b2cd3f0750 => generated 259 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 14:45:14.582723 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:14.650976 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 66/139] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 1 14:45:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 14:45:14.951187 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3801d6-0f54-4d88-b758-8e4f96723658 None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:14.955224 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:15.059506 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3801d6-0f54-4d88-b758-8e4f96723658 None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:15.094524 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7b34d7-45ea-4cb0-90a7-1d89e0e5fa05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:15.095944 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7b34d7-45ea-4cb0-90a7-1d89e0e5fa05 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:15.096434 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7b34d7-45ea-4cb0-90a7-1d89e0e5fa05 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:15.096890 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7b34d7-45ea-4cb0-90a7-1d89e0e5fa05 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:15.098209 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7b34d7-45ea-4cb0-90a7-1d89e0e5fa05 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:15.099948 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7b34d7-45ea-4cb0-90a7-1d89e0e5fa05 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:15.102587 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 74/140] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 1 14:45:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:45:15.108409 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ffa4a3-b4f1-4d29-a3b2-2d002c7d05de None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:15.211531 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ffa4a3-b4f1-4d29-a3b2-2d002c7d05de None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:15.246889 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8684d6cf-54b0-44c1-a050-8f8ec2b8bf04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:15.247854 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8684d6cf-54b0-44c1-a050-8f8ec2b8bf04 None admin] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:15.248357 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8684d6cf-54b0-44c1-a050-8f8ec2b8bf04 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:15.248821 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8684d6cf-54b0-44c1-a050-8f8ec2b8bf04 None admin] PATH_INFO: `/v3/users` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:15.249671 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8684d6cf-54b0-44c1-a050-8f8ec2b8bf04 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:15.251478 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8684d6cf-54b0-44c1-a050-8f8ec2b8bf04 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:15.269339 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:15.284438 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 67/141] 10.222.0.33 () {60 vars in 1132 bytes} [Thu Sep 1 14:45:15 2022] POST /identity/v3/users => generated 283 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:45:15.484282 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf6ce67-af5f-4d8a-8b6d-d16d3fb72b17 None None] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:15.484615 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf6ce67-af5f-4d8a-8b6d-d16d3fb72b17 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:15.484790 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf6ce67-af5f-4d8a-8b6d-d16d3fb72b17 None None] PATH_INFO: `/` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:15.486277 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 75/142] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 1 14:45:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:45:15.491847 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a2533d-33e8-4116-a4c3-916609159668 None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:15.492135 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a2533d-33e8-4116-a4c3-916609159668 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:15.492407 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a2533d-33e8-4116-a4c3-916609159668 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:15.517165 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-03a2533d-33e8-4116-a4c3-916609159668 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:45:15.518377 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-03a2533d-33e8-4116-a4c3-916609159668 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:45:15.588649 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 68/143] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 1 14:45:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 14:45:15.605240 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceadd6a6-6173-42de-bb70-fedcb1e89ca1 None None] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:15.605522 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceadd6a6-6173-42de-bb70-fedcb1e89ca1 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:15.605793 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceadd6a6-6173-42de-bb70-fedcb1e89ca1 None None] PATH_INFO: `/` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:15.607163 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 76/144] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 1 14:45:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:45:15.612327 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d816bc87-76ce-4f2d-aa56-975b966d3d96 None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:15.612616 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d816bc87-76ce-4f2d-aa56-975b966d3d96 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:15.612896 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d816bc87-76ce-4f2d-aa56-975b966d3d96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:15.634506 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-d816bc87-76ce-4f2d-aa56-975b966d3d96 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:45:15.635699 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-d816bc87-76ce-4f2d-aa56-975b966d3d96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:45:15.735366 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 69/145] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 1 14:45:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 14:45:15.893087 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceadd6a6-6173-42de-bb70-fedcb1e89ca1 None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:15.968423 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:16.000028 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceadd6a6-6173-42de-bb70-fedcb1e89ca1 None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:16.036150 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03e7072-7686-4e18-99de-08c710784a22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:16.036780 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03e7072-7686-4e18-99de-08c710784a22 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:16.037072 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03e7072-7686-4e18-99de-08c710784a22 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:16.037315 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03e7072-7686-4e18-99de-08c710784a22 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:16.048364 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03e7072-7686-4e18-99de-08c710784a22 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 14:45:16.053647 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application [None req-c03e7072-7686-4e18-99de-08c710784a22 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 14:45:16.053647 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:45:16.053647 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:45:16.053647 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:45:16.053647 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:45:16.053647 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:45:16.053647 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:45:16.053647 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:45:16.053647 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:45:16.053647 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:45:16.053647 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:45:16.053647 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:45:16.053647 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 14:45:16.053647 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 14:45:16.053647 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 14:45:16.053647 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 14:45:16.054652 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 14:45:16.054652 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 14:45:16.054652 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 14:45:16.054652 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 14:45:16.054652 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:45:16.054652 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:45:16.054652 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:45:16.054652 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:45:16.054652 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:45:16.054652 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:45:16.054652 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:45:16.054652 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application with Lock( Sep 01 14:45:16.054652 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:45:16.054652 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:45:16.054652 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:45:16.054652 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:45:16.056042 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:45:16.056042 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:45:16.056042 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:45:16.056042 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:45:16.056042 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 14:45:16.056042 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 14:45:16.056042 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 14:45:16.056042 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 14:45:16.056042 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 14:45:16.056042 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 14:45:16.056042 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 14:45:16.056042 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Sep 01 14:45:16.056042 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 77/146] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Sep 1 14:45:15 2022] GET /identity/v3/roles/member => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:45:16.060671 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d816bc87-76ce-4f2d-aa56-975b966d3d96 None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:16.118692 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:16.164619 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d816bc87-76ce-4f2d-aa56-975b966d3d96 None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:16.199938 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab1f976-5e25-49c2-acbf-baede1d0bcb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:16.200551 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab1f976-5e25-49c2-acbf-baede1d0bcb3 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:16.200828 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab1f976-5e25-49c2-acbf-baede1d0bcb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:16.201099 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab1f976-5e25-49c2-acbf-baede1d0bcb3 None admin] PATH_INFO: `/v3/roles` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:16.201765 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab1f976-5e25-49c2-acbf-baede1d0bcb3 None admin] RBAC: Adding query filter params (name=member) {{(pid=81341) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:45:16.202063 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab1f976-5e25-49c2-acbf-baede1d0bcb3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:16.203497 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab1f976-5e25-49c2-acbf-baede1d0bcb3 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:16.212843 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 70/147] 10.222.0.33 () {56 vars in 1101 bytes} [Thu Sep 1 14:45:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:45:16.218571 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03e7072-7686-4e18-99de-08c710784a22 None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:16.283553 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab1f976-5e25-49c2-acbf-baede1d0bcb3 None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:16.378464 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03e7072-7686-4e18-99de-08c710784a22 None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:16.426690 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:16.427277 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab1f976-5e25-49c2-acbf-baede1d0bcb3 None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:16.430609 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b8026d-9ac1-4cd6-8e75-f691cd5d7eaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:16.435090 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b8026d-9ac1-4cd6-8e75-f691cd5d7eaa None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:16.435536 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b8026d-9ac1-4cd6-8e75-f691cd5d7eaa None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:16.435982 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b8026d-9ac1-4cd6-8e75-f691cd5d7eaa None admin] PATH_INFO: `/v3/users/4b65298713a24cada6ba75d786076e93` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:16.437456 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b8026d-9ac1-4cd6-8e75-f691cd5d7eaa None admin] RBAC: Authorizing `identity:get_user(user_id=4b65298713a24cada6ba75d786076e93)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:16.439173 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b8026d-9ac1-4cd6-8e75-f691cd5d7eaa None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:16.440911 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 78/148] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Sep 1 14:45:16 2022] GET /identity/v3/users/4b65298713a24cada6ba75d786076e93 => generated 275 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:45:16.450435 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:16.463656 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b8026d-9ac1-4cd6-8e75-f691cd5d7eaa None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:16.471288 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d5eff4-354b-4ad3-8016-389562158001 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:16.472195 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d5eff4-354b-4ad3-8016-389562158001 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:16.472667 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d5eff4-354b-4ad3-8016-389562158001 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:16.473129 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d5eff4-354b-4ad3-8016-389562158001 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:16.483932 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d5eff4-354b-4ad3-8016-389562158001 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 14:45:16.489459 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application [None req-a0d5eff4-354b-4ad3-8016-389562158001 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 14:45:16.489459 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:45:16.489459 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:45:16.489459 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:45:16.489459 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:45:16.489459 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:45:16.489459 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:45:16.489459 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:45:16.489459 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:45:16.489459 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:45:16.489459 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:45:16.489459 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:45:16.489459 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 14:45:16.489459 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 14:45:16.489459 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 14:45:16.489459 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 14:45:16.490355 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 14:45:16.490355 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 14:45:16.490355 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 14:45:16.490355 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 14:45:16.490355 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:45:16.490355 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:45:16.490355 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:45:16.490355 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:45:16.490355 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:45:16.490355 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:45:16.490355 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:45:16.490355 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application with Lock( Sep 01 14:45:16.490355 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:45:16.490355 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:45:16.490355 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:45:16.490355 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:45:16.491731 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:45:16.491731 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:45:16.491731 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:45:16.491731 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:45:16.491731 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 14:45:16.491731 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 14:45:16.491731 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 14:45:16.491731 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 14:45:16.491731 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 14:45:16.491731 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 14:45:16.491731 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 14:45:16.491731 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Sep 01 14:45:16.494328 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 71/149] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Sep 1 14:45:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:45:16.499628 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d5eff4-354b-4ad3-8016-389562158001 None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:16.624906 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d5eff4-354b-4ad3-8016-389562158001 None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:16.677487 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa51c55d-c0d9-4768-be8d-18dace398ed1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:16.679305 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa51c55d-c0d9-4768-be8d-18dace398ed1 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:16.679941 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa51c55d-c0d9-4768-be8d-18dace398ed1 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:16.681844 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa51c55d-c0d9-4768-be8d-18dace398ed1 None admin] PATH_INFO: `/v3/roles` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:16.682862 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa51c55d-c0d9-4768-be8d-18dace398ed1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81341) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:45:16.683380 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa51c55d-c0d9-4768-be8d-18dace398ed1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:16.685948 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa51c55d-c0d9-4768-be8d-18dace398ed1 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:16.705403 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 72/150] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Sep 1 14:45:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:45:16.710981 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa51c55d-c0d9-4768-be8d-18dace398ed1 None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:16.728124 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b8026d-9ac1-4cd6-8e75-f691cd5d7eaa None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:16.794824 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:16.805093 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6028bc8-f923-4275-a748-1315cd2787cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:16.807157 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6028bc8-f923-4275-a748-1315cd2787cc None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:16.807638 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6028bc8-f923-4275-a748-1315cd2787cc None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:16.808115 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6028bc8-f923-4275-a748-1315cd2787cc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:16.926229 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa51c55d-c0d9-4768-be8d-18dace398ed1 None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:16.932840 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6028bc8-f923-4275-a748-1315cd2787cc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:16.934401 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6028bc8-f923-4275-a748-1315cd2787cc None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:16.948572 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:17.013596 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a19d4cc-d42d-4c5b-aab8-56a0fbfeb526 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:17.014269 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a19d4cc-d42d-4c5b-aab8-56a0fbfeb526 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:17.014580 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a19d4cc-d42d-4c5b-aab8-56a0fbfeb526 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:17.014852 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a19d4cc-d42d-4c5b-aab8-56a0fbfeb526 None admin] PATH_INFO: `/v3/users/abbabc07754b486599ec507190d24c97` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:17.016032 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a19d4cc-d42d-4c5b-aab8-56a0fbfeb526 None admin] RBAC: Authorizing `identity:get_user(user_id=abbabc07754b486599ec507190d24c97)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:17.017566 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a19d4cc-d42d-4c5b-aab8-56a0fbfeb526 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:17.019204 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 73/151] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Sep 1 14:45:16 2022] GET /identity/v3/users/abbabc07754b486599ec507190d24c97 => generated 247 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:45:17.044042 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a19d4cc-d42d-4c5b-aab8-56a0fbfeb526 None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:17.057787 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 79/152] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Sep 1 14:45:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 595 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 01 14:45:17.063976 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612580e9-1eb2-4656-bf44-08b1b09ac29f None None] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:17.064264 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612580e9-1eb2-4656-bf44-08b1b09ac29f None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:17.064555 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612580e9-1eb2-4656-bf44-08b1b09ac29f None None] PATH_INFO: `/` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:17.065847 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 80/153] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 1 14:45:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 14:45:17.066579 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612580e9-1eb2-4656-bf44-08b1b09ac29f None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:17.219158 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:17.224190 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612580e9-1eb2-4656-bf44-08b1b09ac29f None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:17.240129 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a19d4cc-d42d-4c5b-aab8-56a0fbfeb526 None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:17.259258 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:17.296449 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c1319d-1659-46f8-90d7-fd5b00786bb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:17.297092 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c1319d-1659-46f8-90d7-fd5b00786bb5 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:17.297373 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c1319d-1659-46f8-90d7-fd5b00786bb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:17.297647 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c1319d-1659-46f8-90d7-fd5b00786bb5 None admin] PATH_INFO: `/v3/projects/13c8e0cc69454dbcadb4dce58c2565b0` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:17.299862 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c1319d-1659-46f8-90d7-fd5b00786bb5 None admin] RBAC: Authorizing `identity:get_project(project_id=13c8e0cc69454dbcadb4dce58c2565b0)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:17.301337 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c1319d-1659-46f8-90d7-fd5b00786bb5 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:17.303259 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 81/154] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Sep 1 14:45:17 2022] GET /identity/v3/projects/13c8e0cc69454dbcadb4dce58c2565b0 => generated 312 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:45:17.308399 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90af6bb-1ba9-4b5a-af5f-d3afad203680 None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:17.308687 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90af6bb-1ba9-4b5a-af5f-d3afad203680 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:17.308956 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90af6bb-1ba9-4b5a-af5f-d3afad203680 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:17.321374 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88293c1-86c3-4460-a57e-5f8f8fd485be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:17.321993 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88293c1-86c3-4460-a57e-5f8f8fd485be None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:17.322293 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88293c1-86c3-4460-a57e-5f8f8fd485be None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:17.322571 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88293c1-86c3-4460-a57e-5f8f8fd485be None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:17.345210 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-d90af6bb-1ba9-4b5a-af5f-d3afad203680 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:45:17.346459 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-d90af6bb-1ba9-4b5a-af5f-d3afad203680 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:45:17.439410 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88293c1-86c3-4460-a57e-5f8f8fd485be None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:17.440959 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88293c1-86c3-4460-a57e-5f8f8fd485be None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:17.487889 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 82/155] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 1 14:45:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 14:45:17.488672 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90af6bb-1ba9-4b5a-af5f-d3afad203680 None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:17.596912 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:17.602830 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90af6bb-1ba9-4b5a-af5f-d3afad203680 None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:17.645858 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badd0dfa-3256-4826-bb6c-1cb7b4cdd5b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:17.646916 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badd0dfa-3256-4826-bb6c-1cb7b4cdd5b1 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:17.647894 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badd0dfa-3256-4826-bb6c-1cb7b4cdd5b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:17.648443 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badd0dfa-3256-4826-bb6c-1cb7b4cdd5b1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:17.649413 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-badd0dfa-3256-4826-bb6c-1cb7b4cdd5b1 None admin] RBAC: Adding query filter params (role.id=5d1feef0e13d4874928bd86258c4e3b2, scope.project.id=13c8e0cc69454dbcadb4dce58c2565b0, user.id=4b65298713a24cada6ba75d786076e93) {{(pid=81342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:45:17.649919 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-badd0dfa-3256-4826-bb6c-1cb7b4cdd5b1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:17.651684 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-badd0dfa-3256-4826-bb6c-1cb7b4cdd5b1 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:17.658179 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 83/156] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Sep 1 14:45:17 2022] GET /identity/v3/role_assignments?user.id=4b65298713a24cada6ba75d786076e93&scope.project.id=13c8e0cc69454dbcadb4dce58c2565b0&role.id=5d1feef0e13d4874928bd86258c4e3b2 => generated 608 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:45:17.659689 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397b147d-c8f3-4cae-b6a9-9cca0082b292 None None] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:17.660182 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397b147d-c8f3-4cae-b6a9-9cca0082b292 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:17.660823 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397b147d-c8f3-4cae-b6a9-9cca0082b292 None None] PATH_INFO: `/` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:17.662272 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 84/157] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 1 14:45:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:45:17.667633 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 74/158] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Sep 1 14:45:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 624 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 14:45:17.668541 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a6b63f-c6e4-4e3e-a4e5-f3f4f0037e18 None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:17.669032 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a6b63f-c6e4-4e3e-a4e5-f3f4f0037e18 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:17.669505 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a6b63f-c6e4-4e3e-a4e5-f3f4f0037e18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:17.675635 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88293c1-86c3-4460-a57e-5f8f8fd485be None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:17.696658 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:17.707714 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-10a6b63f-c6e4-4e3e-a4e5-f3f4f0037e18 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:45:17.709351 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-10a6b63f-c6e4-4e3e-a4e5-f3f4f0037e18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:45:17.817227 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 85/159] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 1 14:45:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 14:45:17.894217 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88293c1-86c3-4460-a57e-5f8f8fd485be None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:17.942337 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620b8d3c-a851-4154-b366-3eec64e8bd53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:17.942992 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620b8d3c-a851-4154-b366-3eec64e8bd53 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:17.943271 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620b8d3c-a851-4154-b366-3eec64e8bd53 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:17.943611 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620b8d3c-a851-4154-b366-3eec64e8bd53 None admin] PATH_INFO: `/v3/projects/1f696f55becc4e879b6d1bac9d93d055` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:17.945356 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620b8d3c-a851-4154-b366-3eec64e8bd53 None admin] RBAC: Authorizing `identity:get_project(project_id=1f696f55becc4e879b6d1bac9d93d055)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:17.947147 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620b8d3c-a851-4154-b366-3eec64e8bd53 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:17.949028 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 75/160] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Sep 1 14:45:17 2022] GET /identity/v3/projects/1f696f55becc4e879b6d1bac9d93d055 => generated 298 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 14:45:17.955028 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a6b63f-c6e4-4e3e-a4e5-f3f4f0037e18 None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:18.008632 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95412e4-db6c-4e46-b2c5-dbe0ecbf73a1 None None] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:18.008632 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95412e4-db6c-4e46-b2c5-dbe0ecbf73a1 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:18.009005 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95412e4-db6c-4e46-b2c5-dbe0ecbf73a1 None None] PATH_INFO: `/` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:18.014269 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 76/161] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 1 14:45:18 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:45:18.017196 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14abbcd5-0765-41ae-b13b-5120d54bc81b None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:18.017723 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14abbcd5-0765-41ae-b13b-5120d54bc81b None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:18.018222 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14abbcd5-0765-41ae-b13b-5120d54bc81b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:18.051299 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-14abbcd5-0765-41ae-b13b-5120d54bc81b None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:45:18.052907 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-14abbcd5-0765-41ae-b13b-5120d54bc81b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:45:18.084357 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:18.096440 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a6b63f-c6e4-4e3e-a4e5-f3f4f0037e18 None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:18.155551 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb33ebf-8ab6-4f56-a97a-273ff6c7a033 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:18.156493 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb33ebf-8ab6-4f56-a97a-273ff6c7a033 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:18.156947 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb33ebf-8ab6-4f56-a97a-273ff6c7a033 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:18.157396 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb33ebf-8ab6-4f56-a97a-273ff6c7a033 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:18.158346 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb33ebf-8ab6-4f56-a97a-273ff6c7a033 None admin] RBAC: Adding query filter params (role.id=2a2adb769e55476b82e5014897e0a9c0, scope.project.id=1f696f55becc4e879b6d1bac9d93d055, user.id=abbabc07754b486599ec507190d24c97) {{(pid=81342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:45:18.158850 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb33ebf-8ab6-4f56-a97a-273ff6c7a033 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:18.160472 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb33ebf-8ab6-4f56-a97a-273ff6c7a033 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:18.165601 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 77/162] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 1 14:45:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 14:45:18.170289 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 86/163] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Sep 1 14:45:17 2022] GET /identity/v3/role_assignments?user.id=abbabc07754b486599ec507190d24c97&scope.project.id=1f696f55becc4e879b6d1bac9d93d055&role.id=2a2adb769e55476b82e5014897e0a9c0 => generated 608 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:45:18.227805 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14abbcd5-0765-41ae-b13b-5120d54bc81b None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:18.237167 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:18.351871 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14abbcd5-0765-41ae-b13b-5120d54bc81b None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:18.391832 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ba3e58-d834-48a1-acd6-cfb208628e06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:18.392475 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ba3e58-d834-48a1-acd6-cfb208628e06 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:18.392755 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ba3e58-d834-48a1-acd6-cfb208628e06 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:18.393025 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ba3e58-d834-48a1-acd6-cfb208628e06 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:18.409015 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ba3e58-d834-48a1-acd6-cfb208628e06 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 14:45:18.414530 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application [None req-25ba3e58-d834-48a1-acd6-cfb208628e06 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 14:45:18.414530 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:45:18.414530 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:45:18.414530 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:45:18.414530 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:45:18.414530 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:45:18.414530 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:45:18.414530 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:45:18.414530 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:45:18.414530 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:45:18.414530 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:45:18.414530 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:45:18.414530 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 14:45:18.414530 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 14:45:18.414530 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 14:45:18.414530 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 14:45:18.415409 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 14:45:18.415409 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 14:45:18.415409 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 14:45:18.415409 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 14:45:18.415409 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:45:18.415409 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:45:18.415409 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:45:18.415409 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:45:18.415409 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:45:18.415409 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:45:18.415409 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:45:18.415409 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application with Lock( Sep 01 14:45:18.415409 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:45:18.415409 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:45:18.415409 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:45:18.415409 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:45:18.416279 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:45:18.416279 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:45:18.416279 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:45:18.416279 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:45:18.416279 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 14:45:18.416279 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 14:45:18.416279 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 14:45:18.416279 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 14:45:18.416279 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 14:45:18.416279 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 14:45:18.416279 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 14:45:18.416279 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Sep 01 14:45:18.416279 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 78/164] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Sep 1 14:45:18 2022] GET /identity/v3/roles/member => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:45:18.423398 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb33ebf-8ab6-4f56-a97a-273ff6c7a033 None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:18.435520 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ba3e58-d834-48a1-acd6-cfb208628e06 None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:18.554613 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:18.582193 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb33ebf-8ab6-4f56-a97a-273ff6c7a033 None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:18.635144 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a004fe-80d3-4098-b4ea-67cf90cfd816 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:18.635702 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a004fe-80d3-4098-b4ea-67cf90cfd816 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:18.635987 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a004fe-80d3-4098-b4ea-67cf90cfd816 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:18.636319 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a004fe-80d3-4098-b4ea-67cf90cfd816 None admin] PATH_INFO: `/v3/roles` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:18.637023 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a004fe-80d3-4098-b4ea-67cf90cfd816 None admin] RBAC: Adding query filter params (name=member) {{(pid=81342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:45:18.637325 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a004fe-80d3-4098-b4ea-67cf90cfd816 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:18.638819 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a004fe-80d3-4098-b4ea-67cf90cfd816 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:18.640958 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ba3e58-d834-48a1-acd6-cfb208628e06 None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:18.647650 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 87/165] 10.222.0.33 () {56 vars in 1101 bytes} [Thu Sep 1 14:45:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:45:18.655275 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a004fe-80d3-4098-b4ea-67cf90cfd816 None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:18.691440 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:18.703020 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341fa3c0-ffdd-43bd-a43e-f6ed807e0397 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:18.703679 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341fa3c0-ffdd-43bd-a43e-f6ed807e0397 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:18.703960 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341fa3c0-ffdd-43bd-a43e-f6ed807e0397 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:18.704231 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341fa3c0-ffdd-43bd-a43e-f6ed807e0397 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:18.739232 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341fa3c0-ffdd-43bd-a43e-f6ed807e0397 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 14:45:18.744608 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application [None req-341fa3c0-ffdd-43bd-a43e-f6ed807e0397 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 14:45:18.744608 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:45:18.744608 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:45:18.744608 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:45:18.744608 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:45:18.744608 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:45:18.744608 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:45:18.744608 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:45:18.744608 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:45:18.744608 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:45:18.744608 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:45:18.744608 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:45:18.744608 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 14:45:18.744608 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 14:45:18.744608 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 14:45:18.744608 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 14:45:18.745517 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 14:45:18.745517 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 14:45:18.745517 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 14:45:18.745517 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 14:45:18.745517 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:45:18.745517 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:45:18.745517 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:45:18.745517 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:45:18.745517 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:45:18.745517 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:45:18.745517 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:45:18.745517 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application with Lock( Sep 01 14:45:18.745517 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:45:18.745517 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:45:18.745517 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:45:18.745517 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:45:18.751033 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:45:18.751033 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:45:18.751033 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:45:18.751033 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:45:18.751033 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 14:45:18.751033 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 14:45:18.751033 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 14:45:18.751033 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 14:45:18.751033 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 14:45:18.751033 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 14:45:18.751033 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 14:45:18.751033 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Sep 01 14:45:18.753146 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 79/166] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Sep 1 14:45:18 2022] GET /identity/v3/roles/reader => generated 84 bytes in 318 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:45:18.754144 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341fa3c0-ffdd-43bd-a43e-f6ed807e0397 None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:18.793142 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a004fe-80d3-4098-b4ea-67cf90cfd816 None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:18.830585 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53175cb5-f6bc-4667-8f3e-8ad2e1f34c79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:18.831225 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53175cb5-f6bc-4667-8f3e-8ad2e1f34c79 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:18.831506 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53175cb5-f6bc-4667-8f3e-8ad2e1f34c79 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:18.831780 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53175cb5-f6bc-4667-8f3e-8ad2e1f34c79 None admin] PATH_INFO: `/v3/users/4b65298713a24cada6ba75d786076e93` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:18.832953 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53175cb5-f6bc-4667-8f3e-8ad2e1f34c79 None admin] RBAC: Authorizing `identity:get_user(user_id=4b65298713a24cada6ba75d786076e93)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:18.834974 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53175cb5-f6bc-4667-8f3e-8ad2e1f34c79 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:18.837407 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 88/167] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Sep 1 14:45:18 2022] GET /identity/v3/users/4b65298713a24cada6ba75d786076e93 => generated 275 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:45:18.838198 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53175cb5-f6bc-4667-8f3e-8ad2e1f34c79 None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:18.934710 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:18.945786 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53175cb5-f6bc-4667-8f3e-8ad2e1f34c79 None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:18.972825 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341fa3c0-ffdd-43bd-a43e-f6ed807e0397 None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:18.999716 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6df664f-d271-408f-a90d-92f917971008 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:19.000825 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6df664f-d271-408f-a90d-92f917971008 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:19.001110 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6df664f-d271-408f-a90d-92f917971008 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:19.001389 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6df664f-d271-408f-a90d-92f917971008 None admin] PATH_INFO: `/v3/roles` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:19.002084 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6df664f-d271-408f-a90d-92f917971008 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:45:19.002422 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6df664f-d271-408f-a90d-92f917971008 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:19.004168 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6df664f-d271-408f-a90d-92f917971008 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:19.013677 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 89/168] 10.222.0.33 () {56 vars in 1101 bytes} [Thu Sep 1 14:45:18 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:45:19.019034 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6df664f-d271-408f-a90d-92f917971008 None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:19.050601 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:19.072393 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa516666-a364-4ced-b5bb-8235ae74b69e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:19.073029 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa516666-a364-4ced-b5bb-8235ae74b69e None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:19.073305 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa516666-a364-4ced-b5bb-8235ae74b69e None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:19.073573 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa516666-a364-4ced-b5bb-8235ae74b69e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:19.101650 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa516666-a364-4ced-b5bb-8235ae74b69e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 14:45:19.107063 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application [None req-fa516666-a364-4ced-b5bb-8235ae74b69e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 14:45:19.107063 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:45:19.107063 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:45:19.107063 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:45:19.107063 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:45:19.107063 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:45:19.107063 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:45:19.107063 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:45:19.107063 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:45:19.107063 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:45:19.107063 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:45:19.107063 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:45:19.107063 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 14:45:19.107063 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 14:45:19.107063 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 14:45:19.107063 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 14:45:19.107955 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 14:45:19.107955 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 14:45:19.107955 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 14:45:19.107955 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 14:45:19.107955 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:45:19.107955 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:45:19.107955 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:45:19.107955 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:45:19.107955 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:45:19.107955 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:45:19.107955 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:45:19.107955 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application with Lock( Sep 01 14:45:19.107955 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:45:19.107955 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:45:19.107955 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:45:19.107955 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:45:19.109153 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:45:19.109153 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:45:19.109153 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:45:19.109153 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:45:19.109153 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 14:45:19.109153 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 14:45:19.109153 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 14:45:19.109153 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 14:45:19.109153 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 14:45:19.109153 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 14:45:19.109153 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 14:45:19.109153 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Sep 01 14:45:19.109153 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 80/169] 10.222.0.33 () {56 vars in 1114 bytes} [Thu Sep 1 14:45:18 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 354 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:45:19.109834 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa516666-a364-4ced-b5bb-8235ae74b69e None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:19.129532 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6df664f-d271-408f-a90d-92f917971008 None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:19.169505 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cda4008-640e-422e-867b-190dc53b408f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:19.170119 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cda4008-640e-422e-867b-190dc53b408f None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:19.170430 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cda4008-640e-422e-867b-190dc53b408f None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:19.170707 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cda4008-640e-422e-867b-190dc53b408f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:19.191181 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:19.248225 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cda4008-640e-422e-867b-190dc53b408f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:19.249701 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cda4008-640e-422e-867b-190dc53b408f None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:19.332575 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa516666-a364-4ced-b5bb-8235ae74b69e None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:19.381943 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ab4288-db84-4771-87bc-1512062a4981 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:19.383047 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ab4288-db84-4771-87bc-1512062a4981 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:19.383345 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ab4288-db84-4771-87bc-1512062a4981 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:19.383619 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ab4288-db84-4771-87bc-1512062a4981 None admin] PATH_INFO: `/v3/users/0159fb4a6f8a45249190b6bae1b464c1` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:19.385042 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ab4288-db84-4771-87bc-1512062a4981 None admin] RBAC: Authorizing `identity:get_user(user_id=0159fb4a6f8a45249190b6bae1b464c1)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:19.387235 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ab4288-db84-4771-87bc-1512062a4981 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:19.389029 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 81/170] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Sep 1 14:45:19 2022] GET /identity/v3/users/0159fb4a6f8a45249190b6bae1b464c1 => generated 289 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:45:19.390105 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ab4288-db84-4771-87bc-1512062a4981 None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:19.410301 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 90/171] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Sep 1 14:45:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 14:45:19.411424 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa69e16-87df-4eef-be00-3b12873bff9c None None] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:19.411751 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa69e16-87df-4eef-be00-3b12873bff9c None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:19.412057 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa69e16-87df-4eef-be00-3b12873bff9c None None] PATH_INFO: `/` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:19.413412 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 91/172] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 1 14:45:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:45:19.414119 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa69e16-87df-4eef-be00-3b12873bff9c None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:19.453272 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:19.487289 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:19.511751 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ab4288-db84-4771-87bc-1512062a4981 None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:19.545213 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa69e16-87df-4eef-be00-3b12873bff9c None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:19.548917 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f984815-eb1c-4cb6-9d7f-5fd629fb80b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:19.549586 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f984815-eb1c-4cb6-9d7f-5fd629fb80b9 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:19.549865 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f984815-eb1c-4cb6-9d7f-5fd629fb80b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:19.550195 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f984815-eb1c-4cb6-9d7f-5fd629fb80b9 None admin] PATH_INFO: `/v3/roles` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:19.550900 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f984815-eb1c-4cb6-9d7f-5fd629fb80b9 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81341) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:45:19.551199 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f984815-eb1c-4cb6-9d7f-5fd629fb80b9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:19.552693 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f984815-eb1c-4cb6-9d7f-5fd629fb80b9 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:19.564553 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 82/173] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Sep 1 14:45:19 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:45:19.565387 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f984815-eb1c-4cb6-9d7f-5fd629fb80b9 None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:19.581002 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4ab492-219a-456f-94da-e078e585c942 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:19.581639 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4ab492-219a-456f-94da-e078e585c942 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:19.581913 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4ab492-219a-456f-94da-e078e585c942 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:19.582181 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4ab492-219a-456f-94da-e078e585c942 None admin] PATH_INFO: `/v3/projects/1f696f55becc4e879b6d1bac9d93d055` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:19.583345 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4ab492-219a-456f-94da-e078e585c942 None admin] RBAC: Authorizing `identity:get_project(project_id=1f696f55becc4e879b6d1bac9d93d055)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:19.584821 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4ab492-219a-456f-94da-e078e585c942 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:19.586634 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 92/174] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Sep 1 14:45:19 2022] GET /identity/v3/projects/1f696f55becc4e879b6d1bac9d93d055 => generated 298 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:45:19.587730 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309d59ae-5a10-45a9-933a-7b6426a500c4 None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:19.587994 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309d59ae-5a10-45a9-933a-7b6426a500c4 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:19.588262 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309d59ae-5a10-45a9-933a-7b6426a500c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:19.606608 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-309d59ae-5a10-45a9-933a-7b6426a500c4 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:45:19.607780 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-309d59ae-5a10-45a9-933a-7b6426a500c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:45:19.679998 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f984815-eb1c-4cb6-9d7f-5fd629fb80b9 None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:19.686022 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 93/175] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 1 14:45:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 14:45:19.687115 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309d59ae-5a10-45a9-933a-7b6426a500c4 None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:19.709448 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:19.735871 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:19.742986 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b535a7-9b22-4a29-b7ec-4ab7b6f4ab23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:19.744244 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b535a7-9b22-4a29-b7ec-4ab7b6f4ab23 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:19.744524 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b535a7-9b22-4a29-b7ec-4ab7b6f4ab23 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:19.744796 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b535a7-9b22-4a29-b7ec-4ab7b6f4ab23 None admin] PATH_INFO: `/v3/projects/1f696f55becc4e879b6d1bac9d93d055` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:19.755105 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b535a7-9b22-4a29-b7ec-4ab7b6f4ab23 None admin] RBAC: Authorizing `identity:get_project(project_id=1f696f55becc4e879b6d1bac9d93d055)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:19.757453 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b535a7-9b22-4a29-b7ec-4ab7b6f4ab23 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:19.759401 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 83/176] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Sep 1 14:45:19 2022] GET /identity/v3/projects/1f696f55becc4e879b6d1bac9d93d055 => generated 298 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:45:19.763943 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b535a7-9b22-4a29-b7ec-4ab7b6f4ab23 None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:19.829620 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309d59ae-5a10-45a9-933a-7b6426a500c4 None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:19.894444 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d60c67-17c8-4e1b-9836-25439e397103 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:19.895314 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d60c67-17c8-4e1b-9836-25439e397103 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:19.895730 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d60c67-17c8-4e1b-9836-25439e397103 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:19.896156 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d60c67-17c8-4e1b-9836-25439e397103 None admin] PATH_INFO: `/v3/users/4b65298713a24cada6ba75d786076e93` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:19.897475 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d60c67-17c8-4e1b-9836-25439e397103 None admin] RBAC: Authorizing `identity:get_user(user_id=4b65298713a24cada6ba75d786076e93)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:19.899116 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d60c67-17c8-4e1b-9836-25439e397103 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:19.900961 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 94/177] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Sep 1 14:45:19 2022] GET /identity/v3/users/4b65298713a24cada6ba75d786076e93 => generated 275 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:45:19.901919 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d60c67-17c8-4e1b-9836-25439e397103 None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:19.926214 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b535a7-9b22-4a29-b7ec-4ab7b6f4ab23 None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:19.960879 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64531eb1-2436-477b-91de-3a85a978a69b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:19.961618 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64531eb1-2436-477b-91de-3a85a978a69b None admin] REQUEST_METHOD: `PUT` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:19.962032 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64531eb1-2436-477b-91de-3a85a978a69b None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:19.962802 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64531eb1-2436-477b-91de-3a85a978a69b None admin] PATH_INFO: `/v3/projects/1f696f55becc4e879b6d1bac9d93d055/users/0159fb4a6f8a45249190b6bae1b464c1/roles/869a5d710fb347a195e8f7b2cd3f0750` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:19.964803 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64531eb1-2436-477b-91de-3a85a978a69b None admin] RBAC: Authorizing `identity:create_grant(project_id=1f696f55becc4e879b6d1bac9d93d055, user_id=0159fb4a6f8a45249190b6bae1b464c1, role_id=869a5d710fb347a195e8f7b2cd3f0750)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:19.966092 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64531eb1-2436-477b-91de-3a85a978a69b None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:19.981621 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 84/178] 10.222.0.33 () {58 vars in 1439 bytes} [Thu Sep 1 14:45:19 2022] PUT /identity/v3/projects/1f696f55becc4e879b6d1bac9d93d055/users/0159fb4a6f8a45249190b6bae1b464c1/roles/869a5d710fb347a195e8f7b2cd3f0750 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:45:19.982831 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64531eb1-2436-477b-91de-3a85a978a69b None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:20.031121 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d60c67-17c8-4e1b-9836-25439e397103 None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:20.038593 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:20.058195 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:20.073368 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3287268e-2295-41e1-9f10-584a1cd356e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:20.074456 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3287268e-2295-41e1-9f10-584a1cd356e5 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:20.074902 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3287268e-2295-41e1-9f10-584a1cd356e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:20.075522 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3287268e-2295-41e1-9f10-584a1cd356e5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:20.076726 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3287268e-2295-41e1-9f10-584a1cd356e5 None admin] RBAC: Adding query filter params (role.id=5d1feef0e13d4874928bd86258c4e3b2, scope.project.id=1f696f55becc4e879b6d1bac9d93d055, user.id=4b65298713a24cada6ba75d786076e93) {{(pid=81342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:45:20.077174 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3287268e-2295-41e1-9f10-584a1cd356e5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:20.079000 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3287268e-2295-41e1-9f10-584a1cd356e5 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:20.086280 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 95/179] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Sep 1 14:45:19 2022] GET /identity/v3/role_assignments?user.id=4b65298713a24cada6ba75d786076e93&scope.project.id=1f696f55becc4e879b6d1bac9d93d055&role.id=5d1feef0e13d4874928bd86258c4e3b2 => generated 608 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:45:20.088120 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44647f2-3ff3-457c-a8df-ddf10449392a None None] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:20.088571 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44647f2-3ff3-457c-a8df-ddf10449392a None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:20.088979 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44647f2-3ff3-457c-a8df-ddf10449392a None None] PATH_INFO: `/` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:20.090498 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 96/180] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 1 14:45:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:45:20.091493 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44647f2-3ff3-457c-a8df-ddf10449392a None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:20.111861 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64531eb1-2436-477b-91de-3a85a978a69b None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:20.152131 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6d94d8-3f7f-4ec3-8c59-d15673d9e93c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:20.153062 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6d94d8-3f7f-4ec3-8c59-d15673d9e93c None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:20.153491 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6d94d8-3f7f-4ec3-8c59-d15673d9e93c None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:20.153923 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6d94d8-3f7f-4ec3-8c59-d15673d9e93c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:20.224523 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44647f2-3ff3-457c-a8df-ddf10449392a None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:20.249129 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6d94d8-3f7f-4ec3-8c59-d15673d9e93c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:20.251218 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6d94d8-3f7f-4ec3-8c59-d15673d9e93c None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:20.274103 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc677013-b853-4ad3-83f3-244989432cf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:20.274909 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc677013-b853-4ad3-83f3-244989432cf3 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:20.275258 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc677013-b853-4ad3-83f3-244989432cf3 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:20.275534 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc677013-b853-4ad3-83f3-244989432cf3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:20.292839 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc677013-b853-4ad3-83f3-244989432cf3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 14:45:20.298726 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application [None req-bc677013-b853-4ad3-83f3-244989432cf3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 14:45:20.298726 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:45:20.298726 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:45:20.298726 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:45:20.298726 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:45:20.298726 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:45:20.298726 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:45:20.298726 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:45:20.298726 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:45:20.298726 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:45:20.298726 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:45:20.298726 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:45:20.298726 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 14:45:20.298726 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 14:45:20.298726 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 14:45:20.298726 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 14:45:20.299621 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 14:45:20.299621 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 14:45:20.299621 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 14:45:20.299621 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 14:45:20.299621 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:45:20.299621 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:45:20.299621 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:45:20.299621 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:45:20.299621 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:45:20.299621 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:45:20.299621 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:45:20.299621 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application with Lock( Sep 01 14:45:20.299621 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:45:20.299621 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:45:20.299621 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:45:20.299621 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:45:20.300436 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:45:20.300436 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:45:20.300436 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:45:20.300436 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:45:20.300436 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 14:45:20.300436 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 14:45:20.300436 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 14:45:20.300436 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 14:45:20.300436 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 14:45:20.300436 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 14:45:20.300436 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 14:45:20.300436 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Sep 01 14:45:20.303083 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 97/181] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Sep 1 14:45:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:45:20.304431 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0377c90d-1196-4c85-a427-10ae5b50fc43 None None] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:20.304713 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0377c90d-1196-4c85-a427-10ae5b50fc43 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:20.304981 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0377c90d-1196-4c85-a427-10ae5b50fc43 None None] PATH_INFO: `/` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:20.306438 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 98/182] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 1 14:45:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:45:20.307627 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b53b3c-1812-4934-a225-8174f6c3b16e None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:20.308075 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b53b3c-1812-4934-a225-8174f6c3b16e None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:20.308511 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b53b3c-1812-4934-a225-8174f6c3b16e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:20.319982 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:20.330646 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-e7b53b3c-1812-4934-a225-8174f6c3b16e None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:45:20.332292 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-e7b53b3c-1812-4934-a225-8174f6c3b16e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:45:20.368117 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:20.389637 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 85/183] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Sep 1 14:45:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 14:45:20.395303 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6d94d8-3f7f-4ec3-8c59-d15673d9e93c None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:20.417352 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 99/184] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 1 14:45:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 14:45:20.418777 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f312b0eb-2f22-46fe-b4e7-96bd4a57563f None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:20.419139 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f312b0eb-2f22-46fe-b4e7-96bd4a57563f None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:20.419504 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f312b0eb-2f22-46fe-b4e7-96bd4a57563f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:20.443150 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-f312b0eb-2f22-46fe-b4e7-96bd4a57563f None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:45:20.444674 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-f312b0eb-2f22-46fe-b4e7-96bd4a57563f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:45:20.502939 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6d94d8-3f7f-4ec3-8c59-d15673d9e93c None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:20.516814 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 100/185] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 1 14:45:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 14:45:20.518455 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f312b0eb-2f22-46fe-b4e7-96bd4a57563f None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:20.544750 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4101acf2-b466-417e-86b2-27f80d117db1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:20.545357 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4101acf2-b466-417e-86b2-27f80d117db1 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:20.545639 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4101acf2-b466-417e-86b2-27f80d117db1 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:20.545914 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4101acf2-b466-417e-86b2-27f80d117db1 None admin] PATH_INFO: `/v3/roles` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:20.546805 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4101acf2-b466-417e-86b2-27f80d117db1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81341) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:45:20.547105 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4101acf2-b466-417e-86b2-27f80d117db1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:20.548517 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4101acf2-b466-417e-86b2-27f80d117db1 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:20.565038 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 86/186] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Sep 1 14:45:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:45:20.574348 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4101acf2-b466-417e-86b2-27f80d117db1 None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:20.602604 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:20.719542 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:20.724039 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f312b0eb-2f22-46fe-b4e7-96bd4a57563f None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:20.748864 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4101acf2-b466-417e-86b2-27f80d117db1 None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:20.762158 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ef8fb6-3c59-46db-976e-0e1a021d00a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:20.763180 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ef8fb6-3c59-46db-976e-0e1a021d00a6 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:20.763650 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ef8fb6-3c59-46db-976e-0e1a021d00a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:20.764100 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ef8fb6-3c59-46db-976e-0e1a021d00a6 None admin] PATH_INFO: `/v3/projects/1f696f55becc4e879b6d1bac9d93d055` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:20.765522 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ef8fb6-3c59-46db-976e-0e1a021d00a6 None admin] RBAC: Authorizing `identity:get_project(project_id=1f696f55becc4e879b6d1bac9d93d055)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:20.767333 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ef8fb6-3c59-46db-976e-0e1a021d00a6 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:20.769255 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 101/187] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Sep 1 14:45:20 2022] GET /identity/v3/projects/1f696f55becc4e879b6d1bac9d93d055 => generated 298 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:45:20.770176 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ef8fb6-3c59-46db-976e-0e1a021d00a6 None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:20.794283 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b99339-f92d-414b-b8cf-b004e59be526 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:20.794283 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b99339-f92d-414b-b8cf-b004e59be526 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:20.794283 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b99339-f92d-414b-b8cf-b004e59be526 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:20.794283 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b99339-f92d-414b-b8cf-b004e59be526 None admin] PATH_INFO: `/v3/users/80eb7b456bec41828b3adc680eabbadb` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:20.811757 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b99339-f92d-414b-b8cf-b004e59be526 None admin] RBAC: Authorizing `identity:get_user(user_id=80eb7b456bec41828b3adc680eabbadb)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:20.813442 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b99339-f92d-414b-b8cf-b004e59be526 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:20.816099 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 87/188] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Sep 1 14:45:20 2022] GET /identity/v3/users/80eb7b456bec41828b3adc680eabbadb => generated 283 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:45:20.817124 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b99339-f92d-414b-b8cf-b004e59be526 None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:20.878700 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ef8fb6-3c59-46db-976e-0e1a021d00a6 None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:20.890515 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:20.913966 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac501ca-5885-4e29-aee6-28f7ae005fd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:20.914636 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac501ca-5885-4e29-aee6-28f7ae005fd7 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:20.914918 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac501ca-5885-4e29-aee6-28f7ae005fd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:20.915189 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac501ca-5885-4e29-aee6-28f7ae005fd7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:20.925548 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac501ca-5885-4e29-aee6-28f7ae005fd7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 14:45:20.928503 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b99339-f92d-414b-b8cf-b004e59be526 None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:20.931293 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application [None req-8ac501ca-5885-4e29-aee6-28f7ae005fd7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 14:45:20.931293 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:45:20.931293 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:45:20.931293 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:45:20.931293 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:45:20.931293 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:45:20.931293 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:45:20.931293 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:45:20.931293 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:45:20.931293 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:45:20.931293 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:45:20.931293 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:45:20.931293 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 14:45:20.931293 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 14:45:20.931293 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 14:45:20.931293 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 14:45:20.932199 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 14:45:20.932199 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 14:45:20.932199 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 14:45:20.932199 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 14:45:20.932199 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:45:20.932199 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:45:20.932199 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:45:20.932199 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:45:20.932199 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:45:20.932199 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:45:20.932199 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:45:20.932199 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application with Lock( Sep 01 14:45:20.932199 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:45:20.932199 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:45:20.932199 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:45:20.932199 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:45:20.933018 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:45:20.933018 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:45:20.933018 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:45:20.933018 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:45:20.933018 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 14:45:20.933018 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 14:45:20.933018 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 14:45:20.933018 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 14:45:20.933018 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 14:45:20.933018 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 14:45:20.933018 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 14:45:20.933018 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Sep 01 14:45:20.933018 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 102/189] 10.222.0.33 () {56 vars in 1114 bytes} [Thu Sep 1 14:45:20 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:45:20.933695 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac501ca-5885-4e29-aee6-28f7ae005fd7 None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:20.962406 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:20.969439 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d894f0-e94a-46c0-9912-15821d47304d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:20.970040 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d894f0-e94a-46c0-9912-15821d47304d None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:20.970351 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d894f0-e94a-46c0-9912-15821d47304d None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:20.970623 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d894f0-e94a-46c0-9912-15821d47304d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:20.971345 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d894f0-e94a-46c0-9912-15821d47304d None admin] RBAC: Adding query filter params (role.id=9996407d588d4a00836c8c50e7f37b1a, scope.project.id=1f696f55becc4e879b6d1bac9d93d055, user.id=4b65298713a24cada6ba75d786076e93) {{(pid=81341) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:45:20.971654 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d894f0-e94a-46c0-9912-15821d47304d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:20.973090 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d894f0-e94a-46c0-9912-15821d47304d None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:20.982279 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 88/190] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Sep 1 14:45:20 2022] GET /identity/v3/role_assignments?user.id=4b65298713a24cada6ba75d786076e93&scope.project.id=1f696f55becc4e879b6d1bac9d93d055&role.id=9996407d588d4a00836c8c50e7f37b1a => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:45:20.982279 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d894f0-e94a-46c0-9912-15821d47304d None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:21.057680 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac501ca-5885-4e29-aee6-28f7ae005fd7 None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:21.102334 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078f9a35-9bf5-40de-b0f4-ab86d68ddc1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:21.103766 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078f9a35-9bf5-40de-b0f4-ab86d68ddc1c None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:21.104265 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078f9a35-9bf5-40de-b0f4-ab86d68ddc1c None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:21.104939 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078f9a35-9bf5-40de-b0f4-ab86d68ddc1c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:21.127860 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d894f0-e94a-46c0-9912-15821d47304d None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:21.167897 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:21.177136 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55afc329-d814-499e-a89d-b26734764e8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:21.177772 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55afc329-d814-499e-a89d-b26734764e8e None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:21.178638 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55afc329-d814-499e-a89d-b26734764e8e None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:21.178918 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55afc329-d814-499e-a89d-b26734764e8e None admin] PATH_INFO: `/v3/domains/default` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:21.180034 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55afc329-d814-499e-a89d-b26734764e8e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:21.181503 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55afc329-d814-499e-a89d-b26734764e8e None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:21.183112 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 89/191] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 1 14:45:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:45:21.183873 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55afc329-d814-499e-a89d-b26734764e8e None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:21.209155 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078f9a35-9bf5-40de-b0f4-ab86d68ddc1c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:21.211743 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078f9a35-9bf5-40de-b0f4-ab86d68ddc1c None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:21.248516 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:21.301370 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55afc329-d814-499e-a89d-b26734764e8e None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:21.332435 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 103/192] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Sep 1 14:45:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 14:45:21.332435 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078f9a35-9bf5-40de-b0f4-ab86d68ddc1c None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:21.346735 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131f5437-9f7c-418d-a688-d4f62d1c7cae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:21.347333 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131f5437-9f7c-418d-a688-d4f62d1c7cae None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:21.347612 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131f5437-9f7c-418d-a688-d4f62d1c7cae None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:21.347886 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131f5437-9f7c-418d-a688-d4f62d1c7cae None admin] PATH_INFO: `/v3/roles` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:21.348547 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131f5437-9f7c-418d-a688-d4f62d1c7cae None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81341) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:45:21.348845 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131f5437-9f7c-418d-a688-d4f62d1c7cae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:21.350274 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131f5437-9f7c-418d-a688-d4f62d1c7cae None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:21.358589 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 90/193] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Sep 1 14:45:21 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:45:21.363120 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131f5437-9f7c-418d-a688-d4f62d1c7cae None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:21.419969 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:21.439704 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078f9a35-9bf5-40de-b0f4-ab86d68ddc1c None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:21.463024 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131f5437-9f7c-418d-a688-d4f62d1c7cae None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:21.467284 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:21.477495 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a015120a-d762-4594-841d-a56a26ca958f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:21.478098 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a015120a-d762-4594-841d-a56a26ca958f None admin] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:21.478415 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a015120a-d762-4594-841d-a56a26ca958f None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:21.478685 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a015120a-d762-4594-841d-a56a26ca958f None admin] PATH_INFO: `/v3/users` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:21.479337 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a015120a-d762-4594-841d-a56a26ca958f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:21.480792 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a015120a-d762-4594-841d-a56a26ca958f None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:21.497797 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5511e46f-b7ab-4931-b944-754fe50dfcaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:21.498472 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5511e46f-b7ab-4931-b944-754fe50dfcaa None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:21.498751 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5511e46f-b7ab-4931-b944-754fe50dfcaa None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:21.499022 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5511e46f-b7ab-4931-b944-754fe50dfcaa None admin] PATH_INFO: `/v3/projects/b817b0f034a64d72914672463eed8f62` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:21.500153 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5511e46f-b7ab-4931-b944-754fe50dfcaa None admin] RBAC: Authorizing `identity:get_project(project_id=b817b0f034a64d72914672463eed8f62)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:21.501621 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5511e46f-b7ab-4931-b944-754fe50dfcaa None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:21.503309 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 91/194] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Sep 1 14:45:21 2022] GET /identity/v3/projects/b817b0f034a64d72914672463eed8f62 => generated 302 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:45:21.508822 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5511e46f-b7ab-4931-b944-754fe50dfcaa None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:21.508936 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 104/195] 10.222.0.33 () {60 vars in 1132 bytes} [Thu Sep 1 14:45:21 2022] POST /identity/v3/users => generated 297 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:45:21.509567 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a015120a-d762-4594-841d-a56a26ca958f None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:21.637244 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a015120a-d762-4594-841d-a56a26ca958f None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:21.645497 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5511e46f-b7ab-4931-b944-754fe50dfcaa None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:21.680214 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c550f5d1-54b8-44ed-ab4b-3b30b12553e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:21.680542 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c550f5d1-54b8-44ed-ab4b-3b30b12553e8 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:21.680817 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c550f5d1-54b8-44ed-ab4b-3b30b12553e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:21.681086 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c550f5d1-54b8-44ed-ab4b-3b30b12553e8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:21.681820 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c550f5d1-54b8-44ed-ab4b-3b30b12553e8 None admin] RBAC: Adding query filter params (role.id=2a2adb769e55476b82e5014897e0a9c0, scope.project.id=b817b0f034a64d72914672463eed8f62, user.id=80eb7b456bec41828b3adc680eabbadb) {{(pid=81342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:45:21.682128 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c550f5d1-54b8-44ed-ab4b-3b30b12553e8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:21.683589 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:21.684452 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c550f5d1-54b8-44ed-ab4b-3b30b12553e8 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:21.694293 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 105/196] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Sep 1 14:45:21 2022] GET /identity/v3/role_assignments?user.id=80eb7b456bec41828b3adc680eabbadb&scope.project.id=b817b0f034a64d72914672463eed8f62&role.id=2a2adb769e55476b82e5014897e0a9c0 => generated 259 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:45:21.701160 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5cad03-bb92-4e0e-9dee-4e7a3e0c0a49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:21.701948 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5cad03-bb92-4e0e-9dee-4e7a3e0c0a49 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:21.702855 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5cad03-bb92-4e0e-9dee-4e7a3e0c0a49 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:21.702855 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5cad03-bb92-4e0e-9dee-4e7a3e0c0a49 None admin] PATH_INFO: `/v3/users/80eb7b456bec41828b3adc680eabbadb` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:21.704354 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5cad03-bb92-4e0e-9dee-4e7a3e0c0a49 None admin] RBAC: Authorizing `identity:get_user(user_id=80eb7b456bec41828b3adc680eabbadb)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:21.705983 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5cad03-bb92-4e0e-9dee-4e7a3e0c0a49 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:21.708054 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 92/197] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Sep 1 14:45:21 2022] GET /identity/v3/users/80eb7b456bec41828b3adc680eabbadb => generated 283 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:45:21.714491 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c550f5d1-54b8-44ed-ab4b-3b30b12553e8 None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:21.773003 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:21.838494 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c550f5d1-54b8-44ed-ab4b-3b30b12553e8 None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:21.877023 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b741dd50-6ec1-44bd-984e-ebb7d3598b9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:21.877652 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b741dd50-6ec1-44bd-984e-ebb7d3598b9e None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:21.877932 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b741dd50-6ec1-44bd-984e-ebb7d3598b9e None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:21.878206 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b741dd50-6ec1-44bd-984e-ebb7d3598b9e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:21.954859 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b741dd50-6ec1-44bd-984e-ebb7d3598b9e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:21.956714 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b741dd50-6ec1-44bd-984e-ebb7d3598b9e None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:22.004332 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:22.012631 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c3cd3a-7b4f-43ff-92c5-c62c4a490eba None None] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:22.012915 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c3cd3a-7b4f-43ff-92c5-c62c4a490eba None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:22.013190 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c3cd3a-7b4f-43ff-92c5-c62c4a490eba None None] PATH_INFO: `/` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:22.014578 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 93/198] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 1 14:45:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:45:22.020021 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4196ff3-5e6f-483d-8e6a-f39ec7305c74 None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:22.020308 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4196ff3-5e6f-483d-8e6a-f39ec7305c74 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:22.020586 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4196ff3-5e6f-483d-8e6a-f39ec7305c74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:22.046483 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-f4196ff3-5e6f-483d-8e6a-f39ec7305c74 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:45:22.047721 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-f4196ff3-5e6f-483d-8e6a-f39ec7305c74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:45:22.071885 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 106/199] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Sep 1 14:45:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 14:45:22.077725 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b741dd50-6ec1-44bd-984e-ebb7d3598b9e None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:22.136004 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 94/200] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 1 14:45:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 14:45:22.195764 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b741dd50-6ec1-44bd-984e-ebb7d3598b9e None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:22.237678 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028184ef-e1e0-4ec7-8833-4635b8122805 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:22.238705 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028184ef-e1e0-4ec7-8833-4635b8122805 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:22.239208 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028184ef-e1e0-4ec7-8833-4635b8122805 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:22.239688 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028184ef-e1e0-4ec7-8833-4635b8122805 None admin] PATH_INFO: `/v3/projects/b817b0f034a64d72914672463eed8f62` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:22.241601 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028184ef-e1e0-4ec7-8833-4635b8122805 None admin] RBAC: Authorizing `identity:get_project(project_id=b817b0f034a64d72914672463eed8f62)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:22.243450 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028184ef-e1e0-4ec7-8833-4635b8122805 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:22.245617 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 107/201] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Sep 1 14:45:22 2022] GET /identity/v3/projects/b817b0f034a64d72914672463eed8f62 => generated 302 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:45:22.251163 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4196ff3-5e6f-483d-8e6a-f39ec7305c74 None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:22.395523 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4196ff3-5e6f-483d-8e6a-f39ec7305c74 None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:22.420009 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:22.438563 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca63c793-7a8b-4285-be30-a39af02834f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:22.439180 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca63c793-7a8b-4285-be30-a39af02834f3 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:22.439457 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca63c793-7a8b-4285-be30-a39af02834f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:22.439727 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca63c793-7a8b-4285-be30-a39af02834f3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:22.440466 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca63c793-7a8b-4285-be30-a39af02834f3 None admin] RBAC: Adding query filter params (role.id=9996407d588d4a00836c8c50e7f37b1a, scope.project.id=b817b0f034a64d72914672463eed8f62, user.id=80eb7b456bec41828b3adc680eabbadb) {{(pid=81341) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:45:22.440770 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca63c793-7a8b-4285-be30-a39af02834f3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:22.442302 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca63c793-7a8b-4285-be30-a39af02834f3 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:22.448525 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 95/202] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Sep 1 14:45:22 2022] GET /identity/v3/role_assignments?user.id=80eb7b456bec41828b3adc680eabbadb&scope.project.id=b817b0f034a64d72914672463eed8f62&role.id=9996407d588d4a00836c8c50e7f37b1a => generated 259 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:45:22.449774 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028184ef-e1e0-4ec7-8833-4635b8122805 None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:22.477606 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:22.610046 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028184ef-e1e0-4ec7-8833-4635b8122805 None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:22.660948 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927afb02-cfb8-4900-ad44-b42fd60cb71f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:22.661699 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927afb02-cfb8-4900-ad44-b42fd60cb71f None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:22.661861 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927afb02-cfb8-4900-ad44-b42fd60cb71f None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:22.662123 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927afb02-cfb8-4900-ad44-b42fd60cb71f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:22.681828 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927afb02-cfb8-4900-ad44-b42fd60cb71f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 14:45:22.687094 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application [None req-927afb02-cfb8-4900-ad44-b42fd60cb71f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 14:45:22.687094 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:45:22.687094 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:45:22.687094 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:45:22.687094 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:45:22.687094 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:45:22.687094 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:45:22.687094 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:45:22.687094 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:45:22.687094 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:45:22.687094 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:45:22.687094 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:45:22.687094 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 14:45:22.687094 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 14:45:22.687094 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 14:45:22.687094 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 14:45:22.687992 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 14:45:22.687992 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 14:45:22.687992 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 14:45:22.687992 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 14:45:22.687992 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:45:22.687992 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:45:22.687992 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:45:22.687992 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:45:22.687992 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:45:22.687992 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:45:22.687992 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:45:22.687992 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application with Lock( Sep 01 14:45:22.687992 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:45:22.687992 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:45:22.687992 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:45:22.687992 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:45:22.688800 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:45:22.688800 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:45:22.688800 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:45:22.688800 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:45:22.688800 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 14:45:22.688800 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 14:45:22.688800 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 14:45:22.688800 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 14:45:22.688800 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 14:45:22.688800 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 14:45:22.688800 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 14:45:22.688800 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Sep 01 14:45:22.688800 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 108/203] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Sep 1 14:45:22 2022] GET /identity/v3/roles/reader => generated 84 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:45:22.694356 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca63c793-7a8b-4285-be30-a39af02834f3 None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:22.862122 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca63c793-7a8b-4285-be30-a39af02834f3 None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:22.908205 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877cc929-dfe8-480c-b25f-f81e3c7bd0e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:22.909190 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877cc929-dfe8-480c-b25f-f81e3c7bd0e5 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:22.909688 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877cc929-dfe8-480c-b25f-f81e3c7bd0e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:22.910155 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877cc929-dfe8-480c-b25f-f81e3c7bd0e5 None admin] PATH_INFO: `/v3/roles` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:22.911053 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877cc929-dfe8-480c-b25f-f81e3c7bd0e5 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81341) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:45:22.911561 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877cc929-dfe8-480c-b25f-f81e3c7bd0e5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:22.913223 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877cc929-dfe8-480c-b25f-f81e3c7bd0e5 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:22.921815 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 96/204] 10.222.0.33 () {56 vars in 1101 bytes} [Thu Sep 1 14:45:22 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:45:22.927581 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927afb02-cfb8-4900-ad44-b42fd60cb71f None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:23.049261 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:23.093586 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927afb02-cfb8-4900-ad44-b42fd60cb71f None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:23.145542 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebe65c5-aaad-4af2-92f3-a743b1856083 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:23.146099 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebe65c5-aaad-4af2-92f3-a743b1856083 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:23.146416 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebe65c5-aaad-4af2-92f3-a743b1856083 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:23.147243 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebe65c5-aaad-4af2-92f3-a743b1856083 None admin] PATH_INFO: `/v3/users/0159fb4a6f8a45249190b6bae1b464c1` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:23.148416 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebe65c5-aaad-4af2-92f3-a743b1856083 None admin] RBAC: Authorizing `identity:get_user(user_id=0159fb4a6f8a45249190b6bae1b464c1)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:23.149885 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebe65c5-aaad-4af2-92f3-a743b1856083 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:23.151821 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 109/205] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Sep 1 14:45:22 2022] GET /identity/v3/users/0159fb4a6f8a45249190b6bae1b464c1 => generated 289 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:45:23.157699 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877cc929-dfe8-480c-b25f-f81e3c7bd0e5 None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:23.230261 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:23.293728 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877cc929-dfe8-480c-b25f-f81e3c7bd0e5 None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:23.341726 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b663023-6142-4c83-af34-0d0787e60de6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:23.342689 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b663023-6142-4c83-af34-0d0787e60de6 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:23.343197 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b663023-6142-4c83-af34-0d0787e60de6 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:23.343669 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b663023-6142-4c83-af34-0d0787e60de6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:23.430867 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b663023-6142-4c83-af34-0d0787e60de6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:23.433100 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b663023-6142-4c83-af34-0d0787e60de6 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:23.486112 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:23.557251 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebe65c5-aaad-4af2-92f3-a743b1856083 None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:23.558564 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 97/206] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Sep 1 14:45:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 14:45:23.733049 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebe65c5-aaad-4af2-92f3-a743b1856083 None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:23.778749 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937060fe-ab66-41ef-8514-b84154fd2ff0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:23.779381 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937060fe-ab66-41ef-8514-b84154fd2ff0 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:23.779660 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937060fe-ab66-41ef-8514-b84154fd2ff0 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:23.779932 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937060fe-ab66-41ef-8514-b84154fd2ff0 None admin] PATH_INFO: `/v3/projects/1f696f55becc4e879b6d1bac9d93d055` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:23.781071 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937060fe-ab66-41ef-8514-b84154fd2ff0 None admin] RBAC: Authorizing `identity:get_project(project_id=1f696f55becc4e879b6d1bac9d93d055)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:23.782625 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937060fe-ab66-41ef-8514-b84154fd2ff0 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:23.784268 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 110/207] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Sep 1 14:45:23 2022] GET /identity/v3/projects/1f696f55becc4e879b6d1bac9d93d055 => generated 298 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:45:23.790825 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b663023-6142-4c83-af34-0d0787e60de6 None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:23.954552 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b663023-6142-4c83-af34-0d0787e60de6 None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:24.002049 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a3491d-0b8d-4618-85a6-4c3ebaadc4ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:24.002604 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a3491d-0b8d-4618-85a6-4c3ebaadc4ea None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:24.002879 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a3491d-0b8d-4618-85a6-4c3ebaadc4ea None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:24.003146 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a3491d-0b8d-4618-85a6-4c3ebaadc4ea None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:24.003869 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a3491d-0b8d-4618-85a6-4c3ebaadc4ea None admin] RBAC: Adding query filter params (role.id=869a5d710fb347a195e8f7b2cd3f0750, scope.project.id=1f696f55becc4e879b6d1bac9d93d055, user.id=0159fb4a6f8a45249190b6bae1b464c1) {{(pid=81341) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:45:24.004170 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a3491d-0b8d-4618-85a6-4c3ebaadc4ea None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:24.005839 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a3491d-0b8d-4618-85a6-4c3ebaadc4ea None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:24.015306 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 98/208] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Sep 1 14:45:23 2022] GET /identity/v3/role_assignments?user.id=0159fb4a6f8a45249190b6bae1b464c1&scope.project.id=1f696f55becc4e879b6d1bac9d93d055&role.id=869a5d710fb347a195e8f7b2cd3f0750 => generated 608 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:45:24.097962 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384f33db-364f-4bc2-9f95-418c85682d17 None None] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:24.098316 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384f33db-364f-4bc2-9f95-418c85682d17 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:24.098539 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384f33db-364f-4bc2-9f95-418c85682d17 None None] PATH_INFO: `/` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:24.100786 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 111/209] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 1 14:45:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:45:24.106644 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128f65af-2a1a-4e6d-9e95-f2c004e9489e None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:24.106949 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128f65af-2a1a-4e6d-9e95-f2c004e9489e None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:24.107193 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128f65af-2a1a-4e6d-9e95-f2c004e9489e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:24.126510 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-128f65af-2a1a-4e6d-9e95-f2c004e9489e None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:45:24.127728 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-128f65af-2a1a-4e6d-9e95-f2c004e9489e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:45:24.138721 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:24.201358 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 99/210] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 1 14:45:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 14:45:24.286291 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1eb02a1-ad62-4db9-a9ac-dd24936cca4a None None] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:24.286666 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1eb02a1-ad62-4db9-a9ac-dd24936cca4a None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:24.287035 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1eb02a1-ad62-4db9-a9ac-dd24936cca4a None None] PATH_INFO: `/` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:24.289390 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 112/211] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 1 14:45:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:45:24.295603 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6367866c-3631-43c6-a01c-ff3db0956be7 None None] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:24.295885 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6367866c-3631-43c6-a01c-ff3db0956be7 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:24.296158 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6367866c-3631-43c6-a01c-ff3db0956be7 None None] PATH_INFO: `/` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:24.297704 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 113/212] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 1 14:45:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:45:24.297878 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e0fe42-4423-4c7a-a7ba-b14de4cd902f None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:24.298200 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e0fe42-4423-4c7a-a7ba-b14de4cd902f None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:24.298506 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e0fe42-4423-4c7a-a7ba-b14de4cd902f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:24.309818 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58893ec-9d88-4053-8649-71b8e10ff6bc None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:24.310113 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58893ec-9d88-4053-8649-71b8e10ff6bc None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:24.310414 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58893ec-9d88-4053-8649-71b8e10ff6bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:24.323349 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-22e0fe42-4423-4c7a-a7ba-b14de4cd902f None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:45:24.324870 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-22e0fe42-4423-4c7a-a7ba-b14de4cd902f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:45:24.331850 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-c58893ec-9d88-4053-8649-71b8e10ff6bc None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:45:24.333037 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-c58893ec-9d88-4053-8649-71b8e10ff6bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:45:24.337671 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:24.402579 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 100/213] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 1 14:45:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 14:45:24.413723 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 114/214] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 1 14:45:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 14:45:24.512359 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e0fe42-4423-4c7a-a7ba-b14de4cd902f None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:24.590464 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:24.616180 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e0fe42-4423-4c7a-a7ba-b14de4cd902f None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:24.651370 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edff7bfd-f08b-4d7c-962e-44a73cd17736 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:24.652384 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edff7bfd-f08b-4d7c-962e-44a73cd17736 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:24.652863 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edff7bfd-f08b-4d7c-962e-44a73cd17736 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:24.653330 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edff7bfd-f08b-4d7c-962e-44a73cd17736 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:24.670260 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edff7bfd-f08b-4d7c-962e-44a73cd17736 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 14:45:24.670260 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application [None req-edff7bfd-f08b-4d7c-962e-44a73cd17736 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 14:45:24.670260 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:45:24.670260 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:45:24.670260 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:45:24.670260 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:45:24.670260 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:45:24.670260 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:45:24.670260 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:45:24.670260 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:45:24.670260 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:45:24.670260 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:45:24.670260 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:45:24.670260 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 14:45:24.671083 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 14:45:24.671083 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 14:45:24.671083 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 14:45:24.671083 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 14:45:24.671083 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 14:45:24.671083 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 14:45:24.671083 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 14:45:24.671083 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:45:24.671083 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:45:24.671083 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:45:24.671083 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:45:24.671083 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:45:24.671083 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:45:24.671083 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:45:24.671083 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application with Lock( Sep 01 14:45:24.671083 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:45:24.671083 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:45:24.671980 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:45:24.671980 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:45:24.671980 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:45:24.671980 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:45:24.671980 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:45:24.671980 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:45:24.671980 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 14:45:24.671980 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 14:45:24.671980 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 14:45:24.671980 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 14:45:24.671980 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 14:45:24.671980 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 14:45:24.671980 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 14:45:24.671980 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Sep 01 14:45:24.671980 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 101/215] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Sep 1 14:45:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:45:24.676571 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58893ec-9d88-4053-8649-71b8e10ff6bc None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:24.695575 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edff7bfd-f08b-4d7c-962e-44a73cd17736 None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:24.784561 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58893ec-9d88-4053-8649-71b8e10ff6bc None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:24.807259 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edff7bfd-f08b-4d7c-962e-44a73cd17736 None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:24.819277 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed432782-e2bb-46c9-8ecb-53873334f2e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:24.819877 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed432782-e2bb-46c9-8ecb-53873334f2e9 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:24.820149 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed432782-e2bb-46c9-8ecb-53873334f2e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:24.820416 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed432782-e2bb-46c9-8ecb-53873334f2e9 None admin] PATH_INFO: `/v3/roles` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:24.821341 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed432782-e2bb-46c9-8ecb-53873334f2e9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:45:24.821635 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed432782-e2bb-46c9-8ecb-53873334f2e9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:24.823021 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed432782-e2bb-46c9-8ecb-53873334f2e9 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:24.831300 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 115/216] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Sep 1 14:45:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:45:24.832063 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed432782-e2bb-46c9-8ecb-53873334f2e9 None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:24.834579 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:24.845652 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e37313-f9f9-4673-83f1-d355c5a2efeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:24.846221 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e37313-f9f9-4673-83f1-d355c5a2efeb None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:24.846522 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e37313-f9f9-4673-83f1-d355c5a2efeb None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:24.846787 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e37313-f9f9-4673-83f1-d355c5a2efeb None admin] PATH_INFO: `/v3/domains/default` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:24.847878 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e37313-f9f9-4673-83f1-d355c5a2efeb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:24.849331 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e37313-f9f9-4673-83f1-d355c5a2efeb None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:24.850917 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 102/217] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 1 14:45:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 14:45:24.855449 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2602493-87d0-46d1-9969-9370a7fde309 None None] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:24.855734 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2602493-87d0-46d1-9969-9370a7fde309 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:24.856032 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2602493-87d0-46d1-9969-9370a7fde309 None None] PATH_INFO: `/` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:24.857280 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 103/218] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 1 14:45:24 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:45:24.857949 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2602493-87d0-46d1-9969-9370a7fde309 None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:24.860556 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:24.935607 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed432782-e2bb-46c9-8ecb-53873334f2e9 None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:24.959472 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2602493-87d0-46d1-9969-9370a7fde309 None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:24.968550 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be99722-55b4-4696-b975-683e2435651c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:24.969184 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be99722-55b4-4696-b975-683e2435651c None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:24.969460 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be99722-55b4-4696-b975-683e2435651c None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:24.969732 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be99722-55b4-4696-b975-683e2435651c None admin] PATH_INFO: `/v3/roles/member` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:24.979825 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be99722-55b4-4696-b975-683e2435651c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 14:45:24.985880 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application [None req-3be99722-55b4-4696-b975-683e2435651c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 14:45:24.985880 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:45:24.985880 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:45:24.985880 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:45:24.985880 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:45:24.985880 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:45:24.985880 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:45:24.985880 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:45:24.985880 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:45:24.985880 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:45:24.985880 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:45:24.985880 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:45:24.985880 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 14:45:24.985880 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 14:45:24.985880 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 14:45:24.985880 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 14:45:24.986815 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 14:45:24.986815 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 14:45:24.986815 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 14:45:24.986815 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 14:45:24.986815 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:45:24.986815 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:45:24.986815 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:45:24.986815 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:45:24.986815 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:45:24.986815 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:45:24.986815 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:45:24.986815 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application with Lock( Sep 01 14:45:24.986815 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:45:24.986815 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:45:24.986815 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:45:24.986815 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:45:24.988135 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:45:24.988135 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:45:24.988135 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:45:24.988135 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:45:24.988135 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 14:45:24.988135 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 14:45:24.988135 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 14:45:24.988135 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 14:45:24.988135 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 14:45:24.988135 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 14:45:24.988135 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 14:45:24.988135 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Sep 01 14:45:24.988135 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 116/219] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Sep 1 14:45:24 2022] GET /identity/v3/roles/member => generated 84 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:45:24.988135 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be99722-55b4-4696-b975-683e2435651c None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:24.996676 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0809c4e8-6c04-42a6-b850-7f1f48dc0089 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:24.997304 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0809c4e8-6c04-42a6-b850-7f1f48dc0089 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:24.997577 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0809c4e8-6c04-42a6-b850-7f1f48dc0089 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:24.997843 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0809c4e8-6c04-42a6-b850-7f1f48dc0089 None admin] PATH_INFO: `/v3/users/80eb7b456bec41828b3adc680eabbadb` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:24.998985 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0809c4e8-6c04-42a6-b850-7f1f48dc0089 None admin] RBAC: Authorizing `identity:get_user(user_id=80eb7b456bec41828b3adc680eabbadb)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:25.000436 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0809c4e8-6c04-42a6-b850-7f1f48dc0089 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:25.001995 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 104/220] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Sep 1 14:45:24 2022] GET /identity/v3/users/80eb7b456bec41828b3adc680eabbadb => generated 283 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:45:25.003108 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be7af08-1992-4dbe-9e50-e09fe104f3df None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:25.003380 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be7af08-1992-4dbe-9e50-e09fe104f3df None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:25.003644 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be7af08-1992-4dbe-9e50-e09fe104f3df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:25.021888 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-2be7af08-1992-4dbe-9e50-e09fe104f3df None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:45:25.023098 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-2be7af08-1992-4dbe-9e50-e09fe104f3df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:45:25.052812 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:25.083488 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:25.088206 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 105/221] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 1 14:45:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 14:45:25.089004 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be7af08-1992-4dbe-9e50-e09fe104f3df None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:25.094428 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be99722-55b4-4696-b975-683e2435651c None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:25.129844 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aca9bae-9d48-40e0-9532-7dea43b50351 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:25.130688 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aca9bae-9d48-40e0-9532-7dea43b50351 None admin] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:25.130975 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aca9bae-9d48-40e0-9532-7dea43b50351 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:25.131251 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aca9bae-9d48-40e0-9532-7dea43b50351 None admin] PATH_INFO: `/v3/users` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:25.131893 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aca9bae-9d48-40e0-9532-7dea43b50351 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:25.133362 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aca9bae-9d48-40e0-9532-7dea43b50351 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:25.158491 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 117/222] 10.222.0.33 () {60 vars in 1132 bytes} [Thu Sep 1 14:45:24 2022] POST /identity/v3/users => generated 297 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:45:25.158491 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aca9bae-9d48-40e0-9532-7dea43b50351 None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:25.201533 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be7af08-1992-4dbe-9e50-e09fe104f3df None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:25.243121 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2258dde1-e392-4f81-9215-f0046de556df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:25.244280 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2258dde1-e392-4f81-9215-f0046de556df None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:25.244855 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2258dde1-e392-4f81-9215-f0046de556df None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:25.245327 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2258dde1-e392-4f81-9215-f0046de556df None admin] PATH_INFO: `/v3/roles` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:25.246251 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2258dde1-e392-4f81-9215-f0046de556df None admin] RBAC: Adding query filter params (name=member) {{(pid=81341) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:45:25.246701 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2258dde1-e392-4f81-9215-f0046de556df None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:25.248438 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2258dde1-e392-4f81-9215-f0046de556df None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:25.259080 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 106/223] 10.222.0.33 () {56 vars in 1101 bytes} [Thu Sep 1 14:45:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:45:25.271499 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2258dde1-e392-4f81-9215-f0046de556df None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:25.309428 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aca9bae-9d48-40e0-9532-7dea43b50351 None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:25.332938 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:25.354886 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:25.358174 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5f515f-e04a-4fcf-b673-8ce85caa90c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:25.360747 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5f515f-e04a-4fcf-b673-8ce85caa90c4 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:25.360747 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5f515f-e04a-4fcf-b673-8ce85caa90c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:25.360747 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5f515f-e04a-4fcf-b673-8ce85caa90c4 None admin] PATH_INFO: `/v3/projects/b817b0f034a64d72914672463eed8f62` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:25.360747 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5f515f-e04a-4fcf-b673-8ce85caa90c4 None admin] RBAC: Authorizing `identity:get_project(project_id=b817b0f034a64d72914672463eed8f62)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:25.362510 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5f515f-e04a-4fcf-b673-8ce85caa90c4 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:25.363514 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 118/224] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Sep 1 14:45:25 2022] GET /identity/v3/projects/b817b0f034a64d72914672463eed8f62 => generated 302 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:45:25.364263 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5f515f-e04a-4fcf-b673-8ce85caa90c4 None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:25.401710 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2258dde1-e392-4f81-9215-f0046de556df None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:25.444823 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1103064b-5522-4548-9904-85be61fd343c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:25.445749 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1103064b-5522-4548-9904-85be61fd343c None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:25.446323 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1103064b-5522-4548-9904-85be61fd343c None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:25.446601 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1103064b-5522-4548-9904-85be61fd343c None admin] PATH_INFO: `/v3/users/ea5aa8fb75dc489ea6e3bdbcd21263a5` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:25.462932 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1103064b-5522-4548-9904-85be61fd343c None admin] RBAC: Authorizing `identity:get_user(user_id=ea5aa8fb75dc489ea6e3bdbcd21263a5)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:25.464433 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1103064b-5522-4548-9904-85be61fd343c None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:25.466410 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 107/225] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Sep 1 14:45:25 2022] GET /identity/v3/users/ea5aa8fb75dc489ea6e3bdbcd21263a5 => generated 297 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:45:25.466933 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1103064b-5522-4548-9904-85be61fd343c None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:25.490002 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5f515f-e04a-4fcf-b673-8ce85caa90c4 None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:25.537892 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125848a7-cd83-4ad7-9439-8467cf29343a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:25.538575 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125848a7-cd83-4ad7-9439-8467cf29343a None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:25.538857 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125848a7-cd83-4ad7-9439-8467cf29343a None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:25.540404 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125848a7-cd83-4ad7-9439-8467cf29343a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:25.559079 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125848a7-cd83-4ad7-9439-8467cf29343a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 14:45:25.566305 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application [None req-125848a7-cd83-4ad7-9439-8467cf29343a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 14:45:25.566305 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:45:25.566305 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:45:25.566305 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:45:25.566305 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:45:25.566305 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:45:25.566305 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:45:25.566305 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:45:25.566305 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:45:25.566305 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:45:25.566305 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:45:25.566305 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:45:25.566305 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 14:45:25.566305 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 14:45:25.566305 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 14:45:25.566305 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 14:45:25.567238 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 14:45:25.567238 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 14:45:25.567238 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 14:45:25.567238 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 14:45:25.567238 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:45:25.567238 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:45:25.567238 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:45:25.567238 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:45:25.567238 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:45:25.567238 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:45:25.567238 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:45:25.567238 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application with Lock( Sep 01 14:45:25.567238 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:45:25.567238 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:45:25.567238 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:45:25.567238 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:45:25.568063 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:45:25.568063 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:45:25.568063 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:45:25.568063 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:45:25.568063 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 14:45:25.568063 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 14:45:25.568063 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 14:45:25.568063 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 14:45:25.568063 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 14:45:25.568063 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 14:45:25.568063 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 14:45:25.568063 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Sep 01 14:45:25.568063 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 119/226] 10.222.0.33 () {56 vars in 1114 bytes} [Thu Sep 1 14:45:25 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:45:25.568786 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125848a7-cd83-4ad7-9439-8467cf29343a None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:25.593590 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1103064b-5522-4548-9904-85be61fd343c None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:25.619245 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:25.625317 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:25.637112 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1fd3b6-e1dc-4cc0-ae06-ff98fd60547d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:25.638097 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1fd3b6-e1dc-4cc0-ae06-ff98fd60547d None admin] REQUEST_METHOD: `PUT` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:25.638632 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1fd3b6-e1dc-4cc0-ae06-ff98fd60547d None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:25.639207 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1fd3b6-e1dc-4cc0-ae06-ff98fd60547d None admin] PATH_INFO: `/v3/projects/b817b0f034a64d72914672463eed8f62/users/80eb7b456bec41828b3adc680eabbadb/roles/2a2adb769e55476b82e5014897e0a9c0` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:25.641464 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1fd3b6-e1dc-4cc0-ae06-ff98fd60547d None admin] RBAC: Authorizing `identity:create_grant(project_id=b817b0f034a64d72914672463eed8f62, user_id=80eb7b456bec41828b3adc680eabbadb, role_id=2a2adb769e55476b82e5014897e0a9c0)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:25.643427 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1fd3b6-e1dc-4cc0-ae06-ff98fd60547d None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:25.669005 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 108/227] 10.222.0.33 () {58 vars in 1439 bytes} [Thu Sep 1 14:45:25 2022] PUT /identity/v3/projects/b817b0f034a64d72914672463eed8f62/users/80eb7b456bec41828b3adc680eabbadb/roles/2a2adb769e55476b82e5014897e0a9c0 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:45:25.670497 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1fd3b6-e1dc-4cc0-ae06-ff98fd60547d None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:25.691716 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125848a7-cd83-4ad7-9439-8467cf29343a None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:25.736427 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95deab5d-1da9-48cf-9026-c33cf250af6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:25.737905 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95deab5d-1da9-48cf-9026-c33cf250af6f None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:25.738373 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95deab5d-1da9-48cf-9026-c33cf250af6f None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:25.739082 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95deab5d-1da9-48cf-9026-c33cf250af6f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:25.801307 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1fd3b6-e1dc-4cc0-ae06-ff98fd60547d None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:25.838273 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95deab5d-1da9-48cf-9026-c33cf250af6f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:25.839969 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95deab5d-1da9-48cf-9026-c33cf250af6f None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:25.847093 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2e975d-3c07-41a0-befc-24f7497bce13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:25.847922 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2e975d-3c07-41a0-befc-24f7497bce13 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:25.848368 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2e975d-3c07-41a0-befc-24f7497bce13 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:25.848779 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2e975d-3c07-41a0-befc-24f7497bce13 None admin] PATH_INFO: `/v3/roles` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:25.850170 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2e975d-3c07-41a0-befc-24f7497bce13 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81341) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:45:25.850743 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2e975d-3c07-41a0-befc-24f7497bce13 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:25.852751 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2e975d-3c07-41a0-befc-24f7497bce13 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:25.865233 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 109/228] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Sep 1 14:45:25 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:45:25.870065 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2e975d-3c07-41a0-befc-24f7497bce13 None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:25.915439 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:25.949804 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:25.987044 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 120/229] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Sep 1 14:45:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 14:45:25.992582 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95deab5d-1da9-48cf-9026-c33cf250af6f None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:26.014205 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2e975d-3c07-41a0-befc-24f7497bce13 None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:26.068177 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a053a8-bb75-4d51-9492-623bacca1fa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:26.069388 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a053a8-bb75-4d51-9492-623bacca1fa8 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:26.069643 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a053a8-bb75-4d51-9492-623bacca1fa8 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:26.069916 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a053a8-bb75-4d51-9492-623bacca1fa8 None admin] PATH_INFO: `/v3/users/80eb7b456bec41828b3adc680eabbadb` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:26.071469 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a053a8-bb75-4d51-9492-623bacca1fa8 None admin] RBAC: Authorizing `identity:get_user(user_id=80eb7b456bec41828b3adc680eabbadb)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:26.072935 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a053a8-bb75-4d51-9492-623bacca1fa8 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:26.076764 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 110/230] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Sep 1 14:45:25 2022] GET /identity/v3/users/80eb7b456bec41828b3adc680eabbadb => generated 283 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:45:26.083401 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a053a8-bb75-4d51-9492-623bacca1fa8 None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:26.126716 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95deab5d-1da9-48cf-9026-c33cf250af6f None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:26.167161 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649a2937-64b5-4c0a-a642-334142580f0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:26.167710 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649a2937-64b5-4c0a-a642-334142580f0b None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:26.167981 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649a2937-64b5-4c0a-a642-334142580f0b None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:26.168244 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649a2937-64b5-4c0a-a642-334142580f0b None admin] PATH_INFO: `/v3/projects/b817b0f034a64d72914672463eed8f62` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:26.169366 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649a2937-64b5-4c0a-a642-334142580f0b None admin] RBAC: Authorizing `identity:get_project(project_id=b817b0f034a64d72914672463eed8f62)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:26.170844 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649a2937-64b5-4c0a-a642-334142580f0b None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:26.172760 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 121/231] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Sep 1 14:45:25 2022] GET /identity/v3/projects/b817b0f034a64d72914672463eed8f62 => generated 302 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 14:45:26.185032 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649a2937-64b5-4c0a-a642-334142580f0b None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:26.208160 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:26.287062 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649a2937-64b5-4c0a-a642-334142580f0b None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:26.322604 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d2f84b-3db0-4d12-9363-7551480db08c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:26.323213 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d2f84b-3db0-4d12-9363-7551480db08c None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:26.323487 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d2f84b-3db0-4d12-9363-7551480db08c None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:26.323756 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d2f84b-3db0-4d12-9363-7551480db08c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:26.324483 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d2f84b-3db0-4d12-9363-7551480db08c None admin] RBAC: Adding query filter params (role.id=5d1feef0e13d4874928bd86258c4e3b2, scope.project.id=b817b0f034a64d72914672463eed8f62, user.id=ea5aa8fb75dc489ea6e3bdbcd21263a5) {{(pid=81342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:45:26.324897 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d2f84b-3db0-4d12-9363-7551480db08c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:26.326368 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d2f84b-3db0-4d12-9363-7551480db08c None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:26.332081 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 122/232] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Sep 1 14:45:26 2022] GET /identity/v3/role_assignments?user.id=ea5aa8fb75dc489ea6e3bdbcd21263a5&scope.project.id=b817b0f034a64d72914672463eed8f62&role.id=5d1feef0e13d4874928bd86258c4e3b2 => generated 259 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 14:45:26.360702 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a053a8-bb75-4d51-9492-623bacca1fa8 None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:26.383421 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:26.425710 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a7174d-bc07-4b8c-a4d8-09d9843341a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:26.426300 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a7174d-bc07-4b8c-a4d8-09d9843341a2 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:26.426578 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a7174d-bc07-4b8c-a4d8-09d9843341a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:26.426866 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a7174d-bc07-4b8c-a4d8-09d9843341a2 None admin] PATH_INFO: `/v3/projects/b817b0f034a64d72914672463eed8f62` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:26.431083 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a7174d-bc07-4b8c-a4d8-09d9843341a2 None admin] RBAC: Authorizing `identity:get_project(project_id=b817b0f034a64d72914672463eed8f62)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:26.432608 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a7174d-bc07-4b8c-a4d8-09d9843341a2 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:26.434681 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 111/233] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Sep 1 14:45:26 2022] GET /identity/v3/projects/b817b0f034a64d72914672463eed8f62 => generated 302 bytes in 351 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:45:26.440200 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d2f84b-3db0-4d12-9363-7551480db08c None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:26.530668 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:26.550105 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d2f84b-3db0-4d12-9363-7551480db08c None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:26.587894 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9518839-50d3-4320-a3a3-4f5077589a72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:26.588562 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9518839-50d3-4320-a3a3-4f5077589a72 None admin] REQUEST_METHOD: `PUT` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:26.588838 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9518839-50d3-4320-a3a3-4f5077589a72 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:26.589106 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9518839-50d3-4320-a3a3-4f5077589a72 None admin] PATH_INFO: `/v3/projects/b817b0f034a64d72914672463eed8f62/users/80eb7b456bec41828b3adc680eabbadb/roles/9996407d588d4a00836c8c50e7f37b1a` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:26.591305 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9518839-50d3-4320-a3a3-4f5077589a72 None admin] RBAC: Authorizing `identity:create_grant(project_id=b817b0f034a64d72914672463eed8f62, user_id=80eb7b456bec41828b3adc680eabbadb, role_id=9996407d588d4a00836c8c50e7f37b1a)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:26.592765 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9518839-50d3-4320-a3a3-4f5077589a72 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:26.608966 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 123/234] 10.222.0.33 () {58 vars in 1439 bytes} [Thu Sep 1 14:45:26 2022] PUT /identity/v3/projects/b817b0f034a64d72914672463eed8f62/users/80eb7b456bec41828b3adc680eabbadb/roles/9996407d588d4a00836c8c50e7f37b1a => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:45:27.570368 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4da2a9-108c-4612-b97c-3130b8b7624d None None] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:27.570889 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4da2a9-108c-4612-b97c-3130b8b7624d None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:27.571481 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4da2a9-108c-4612-b97c-3130b8b7624d None None] PATH_INFO: `/` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:27.573173 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 112/235] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 1 14:45:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:45:27.578291 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbf58bc-568c-442e-85c1-635ca251fd15 None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:27.578687 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbf58bc-568c-442e-85c1-635ca251fd15 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:27.579016 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbf58bc-568c-442e-85c1-635ca251fd15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:27.597844 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-fdbf58bc-568c-442e-85c1-635ca251fd15 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:45:27.599211 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-fdbf58bc-568c-442e-85c1-635ca251fd15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:45:27.667482 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 124/236] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 1 14:45:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 14:45:27.753004 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad575a62-3556-4774-8386-956e065d8d1a None None] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:27.753344 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad575a62-3556-4774-8386-956e065d8d1a None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:27.753553 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad575a62-3556-4774-8386-956e065d8d1a None None] PATH_INFO: `/` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:27.755197 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 113/237] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 1 14:45:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:45:27.767759 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c788b5a-c028-4c17-87e5-aa2eb04e5517 None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:27.768040 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c788b5a-c028-4c17-87e5-aa2eb04e5517 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:27.768308 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c788b5a-c028-4c17-87e5-aa2eb04e5517 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:27.792203 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-6c788b5a-c028-4c17-87e5-aa2eb04e5517 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:45:27.793374 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-6c788b5a-c028-4c17-87e5-aa2eb04e5517 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:45:27.856800 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:27.862748 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 125/238] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 1 14:45:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 14:45:27.994557 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad575a62-3556-4774-8386-956e065d8d1a None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:28.112175 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad575a62-3556-4774-8386-956e065d8d1a None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:28.147073 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80ca7f8-997a-43e8-b825-e93f154dc07a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:28.147704 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80ca7f8-997a-43e8-b825-e93f154dc07a None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:28.147981 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80ca7f8-997a-43e8-b825-e93f154dc07a None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:28.148244 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80ca7f8-997a-43e8-b825-e93f154dc07a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:28.163601 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80ca7f8-997a-43e8-b825-e93f154dc07a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 14:45:28.168837 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application [None req-e80ca7f8-997a-43e8-b825-e93f154dc07a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 14:45:28.168837 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:45:28.168837 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:45:28.168837 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:45:28.168837 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:45:28.168837 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:45:28.168837 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:45:28.168837 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:45:28.168837 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:45:28.168837 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:45:28.168837 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:45:28.168837 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:45:28.168837 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 14:45:28.168837 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 14:45:28.168837 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 14:45:28.168837 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 14:45:28.169732 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 14:45:28.169732 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 14:45:28.169732 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 14:45:28.169732 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 14:45:28.169732 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:45:28.169732 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:45:28.169732 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:45:28.169732 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:45:28.169732 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:45:28.169732 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:45:28.169732 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:45:28.169732 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application with Lock( Sep 01 14:45:28.169732 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:45:28.169732 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:45:28.169732 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:45:28.169732 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:45:28.170590 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:45:28.170590 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:45:28.170590 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:45:28.170590 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:45:28.170590 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 14:45:28.170590 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 14:45:28.170590 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 14:45:28.170590 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 14:45:28.170590 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 14:45:28.170590 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 14:45:28.170590 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 14:45:28.170590 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Sep 01 14:45:28.170590 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 114/239] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Sep 1 14:45:27 2022] GET /identity/v3/roles/reader => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:45:28.175672 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c788b5a-c028-4c17-87e5-aa2eb04e5517 None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:28.277737 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c788b5a-c028-4c17-87e5-aa2eb04e5517 None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:28.291488 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8e4c43-86da-4afb-b0ff-abe92a7b4cc1 None None] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:28.291764 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8e4c43-86da-4afb-b0ff-abe92a7b4cc1 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:28.292028 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8e4c43-86da-4afb-b0ff-abe92a7b4cc1 None None] PATH_INFO: `/` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:28.293350 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 115/240] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 1 14:45:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 14:45:28.298501 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af54c5fe-5b5a-4de8-97ee-31d95cb08450 None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:28.298784 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af54c5fe-5b5a-4de8-97ee-31d95cb08450 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:28.299055 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af54c5fe-5b5a-4de8-97ee-31d95cb08450 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:28.318359 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a7b7c9-e5eb-4588-a432-500c268bc504 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:28.319008 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a7b7c9-e5eb-4588-a432-500c268bc504 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:28.319281 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a7b7c9-e5eb-4588-a432-500c268bc504 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:28.319549 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a7b7c9-e5eb-4588-a432-500c268bc504 None admin] PATH_INFO: `/v3/roles` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:28.320276 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a7b7c9-e5eb-4588-a432-500c268bc504 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:45:28.321362 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a7b7c9-e5eb-4588-a432-500c268bc504 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:28.322797 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a7b7c9-e5eb-4588-a432-500c268bc504 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:28.331777 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 126/241] 10.222.0.33 () {56 vars in 1101 bytes} [Thu Sep 1 14:45:28 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:45:28.332768 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-af54c5fe-5b5a-4de8-97ee-31d95cb08450 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:45:28.333986 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-af54c5fe-5b5a-4de8-97ee-31d95cb08450 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:45:28.340435 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a7b7c9-e5eb-4588-a432-500c268bc504 None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:28.371276 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:28.411542 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:28.416921 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 116/242] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 1 14:45:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 14:45:28.417889 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af54c5fe-5b5a-4de8-97ee-31d95cb08450 None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:28.456603 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a7b7c9-e5eb-4588-a432-500c268bc504 None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:28.492949 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603e3c17-811d-4e94-bcf0-e7477195bb10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:28.493642 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603e3c17-811d-4e94-bcf0-e7477195bb10 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:28.493929 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603e3c17-811d-4e94-bcf0-e7477195bb10 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:28.494202 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603e3c17-811d-4e94-bcf0-e7477195bb10 None admin] PATH_INFO: `/v3/users/67376baab7bd4cfa8a2e01f91535389e` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:28.509357 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603e3c17-811d-4e94-bcf0-e7477195bb10 None admin] RBAC: Authorizing `identity:get_user(user_id=67376baab7bd4cfa8a2e01f91535389e)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:28.511006 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603e3c17-811d-4e94-bcf0-e7477195bb10 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:28.513004 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 127/243] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Sep 1 14:45:28 2022] GET /identity/v3/users/67376baab7bd4cfa8a2e01f91535389e => generated 297 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:45:28.519678 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603e3c17-811d-4e94-bcf0-e7477195bb10 None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:28.562433 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af54c5fe-5b5a-4de8-97ee-31d95cb08450 None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:28.635924 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95d29fb-0bae-401a-bb97-64cebe183018 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:28.636528 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95d29fb-0bae-401a-bb97-64cebe183018 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:28.636841 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95d29fb-0bae-401a-bb97-64cebe183018 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:28.637112 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95d29fb-0bae-401a-bb97-64cebe183018 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:28.643106 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95d29fb-0bae-401a-bb97-64cebe183018 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:28.644688 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95d29fb-0bae-401a-bb97-64cebe183018 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:28.645867 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603e3c17-811d-4e94-bcf0-e7477195bb10 None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:28.647287 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 117/244] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 1 14:45:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:45:28.656478 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95d29fb-0bae-401a-bb97-64cebe183018 None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:28.682697 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:28.695739 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16cfc92-3fba-4425-9e69-2c6e6c8a198c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:28.696906 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16cfc92-3fba-4425-9e69-2c6e6c8a198c None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:28.697477 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16cfc92-3fba-4425-9e69-2c6e6c8a198c None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:28.697943 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16cfc92-3fba-4425-9e69-2c6e6c8a198c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:28.728757 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:28.772544 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16cfc92-3fba-4425-9e69-2c6e6c8a198c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:28.774438 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16cfc92-3fba-4425-9e69-2c6e6c8a198c None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:28.832724 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95d29fb-0bae-401a-bb97-64cebe183018 None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:28.869241 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d94e727-50e2-4e69-8ca3-5dba1f2f3017 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:28.869840 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d94e727-50e2-4e69-8ca3-5dba1f2f3017 None admin] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:28.870113 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d94e727-50e2-4e69-8ca3-5dba1f2f3017 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:28.870411 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d94e727-50e2-4e69-8ca3-5dba1f2f3017 None admin] PATH_INFO: `/v3/users` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:28.871170 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d94e727-50e2-4e69-8ca3-5dba1f2f3017 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:28.872736 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d94e727-50e2-4e69-8ca3-5dba1f2f3017 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:28.889139 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 128/245] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Sep 1 14:45:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 01 14:45:28.895397 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55de04b9-b73b-4351-8f3c-db01a1a8d460 None None] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:28.895688 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55de04b9-b73b-4351-8f3c-db01a1a8d460 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:28.896003 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 118/246] 10.222.0.33 () {60 vars in 1132 bytes} [Thu Sep 1 14:45:28 2022] POST /identity/v3/users => generated 293 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:45:28.896109 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55de04b9-b73b-4351-8f3c-db01a1a8d460 None None] PATH_INFO: `/` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:28.897387 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d94e727-50e2-4e69-8ca3-5dba1f2f3017 None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:28.898004 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 129/247] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 1 14:45:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:45:28.898631 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55de04b9-b73b-4351-8f3c-db01a1a8d460 None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:28.932873 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:29.034169 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:29.037649 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55de04b9-b73b-4351-8f3c-db01a1a8d460 None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:29.049010 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d94e727-50e2-4e69-8ca3-5dba1f2f3017 None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:29.077165 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba2fde7-7b7c-45be-a0de-69f43b3e0acf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:29.077798 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba2fde7-7b7c-45be-a0de-69f43b3e0acf None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:29.078075 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba2fde7-7b7c-45be-a0de-69f43b3e0acf None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:29.078376 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba2fde7-7b7c-45be-a0de-69f43b3e0acf None admin] PATH_INFO: `/v3/projects/b817b0f034a64d72914672463eed8f62` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:29.079490 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba2fde7-7b7c-45be-a0de-69f43b3e0acf None admin] RBAC: Authorizing `identity:get_project(project_id=b817b0f034a64d72914672463eed8f62)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:29.080937 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba2fde7-7b7c-45be-a0de-69f43b3e0acf None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:29.082755 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 130/248] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Sep 1 14:45:28 2022] GET /identity/v3/projects/b817b0f034a64d72914672463eed8f62 => generated 302 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:45:29.083688 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0bf475-ac43-4337-b4fa-c4fd66c5e029 None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:29.084000 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0bf475-ac43-4337-b4fa-c4fd66c5e029 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:29.084792 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0bf475-ac43-4337-b4fa-c4fd66c5e029 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:29.085241 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec82374-4ecf-4db0-b012-b99452adbedf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:29.085880 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec82374-4ecf-4db0-b012-b99452adbedf None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:29.086163 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec82374-4ecf-4db0-b012-b99452adbedf None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:29.086511 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec82374-4ecf-4db0-b012-b99452adbedf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:29.097505 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec82374-4ecf-4db0-b012-b99452adbedf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 14:45:29.102932 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application [None req-aec82374-4ecf-4db0-b012-b99452adbedf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 14:45:29.102932 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:45:29.102932 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:45:29.102932 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:45:29.102932 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:45:29.102932 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:45:29.102932 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:45:29.102932 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:45:29.102932 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:45:29.102932 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:45:29.102932 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:45:29.102932 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:45:29.102932 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 14:45:29.102932 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 14:45:29.102932 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 14:45:29.102932 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 14:45:29.103848 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 14:45:29.103848 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 14:45:29.103848 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 14:45:29.103848 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 14:45:29.103848 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:45:29.103848 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:45:29.103848 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:45:29.103848 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:45:29.103848 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:45:29.103848 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:45:29.103848 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:45:29.103848 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application with Lock( Sep 01 14:45:29.103848 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:45:29.103848 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:45:29.103848 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:45:29.103848 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:45:29.104670 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:45:29.104670 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:45:29.104670 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:45:29.104670 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:45:29.104670 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 14:45:29.104670 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 14:45:29.104670 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 14:45:29.104670 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 14:45:29.104670 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 14:45:29.104670 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 14:45:29.104670 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 14:45:29.104670 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Sep 01 14:45:29.104670 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 119/249] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Sep 1 14:45:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:45:29.105372 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0753ef4-406c-4fc3-92a1-dd0d3b13bfe4 None None] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:29.106080 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0753ef4-406c-4fc3-92a1-dd0d3b13bfe4 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:29.106080 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0753ef4-406c-4fc3-92a1-dd0d3b13bfe4 None None] PATH_INFO: `/` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:29.107150 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 120/250] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 1 14:45:29 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:45:29.107250 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-da0bf475-ac43-4337-b4fa-c4fd66c5e029 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:45:29.108381 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-da0bf475-ac43-4337-b4fa-c4fd66c5e029 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:45:29.112324 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0753ef4-406c-4fc3-92a1-dd0d3b13bfe4 None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:29.196349 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 131/251] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 1 14:45:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 14:45:29.197507 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0bf475-ac43-4337-b4fa-c4fd66c5e029 None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:29.220458 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:29.248356 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0753ef4-406c-4fc3-92a1-dd0d3b13bfe4 None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:29.290564 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8fcd49-3059-425d-b96e-c74b8187fe17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:29.291130 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8fcd49-3059-425d-b96e-c74b8187fe17 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:29.291419 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8fcd49-3059-425d-b96e-c74b8187fe17 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:29.291690 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8fcd49-3059-425d-b96e-c74b8187fe17 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:29.293042 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8fcd49-3059-425d-b96e-c74b8187fe17 None admin] RBAC: Adding query filter params (role.id=869a5d710fb347a195e8f7b2cd3f0750, scope.project.id=b817b0f034a64d72914672463eed8f62, user.id=67376baab7bd4cfa8a2e01f91535389e) {{(pid=81341) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:45:29.293042 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8fcd49-3059-425d-b96e-c74b8187fe17 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:29.294310 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8fcd49-3059-425d-b96e-c74b8187fe17 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:29.300268 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 121/252] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Sep 1 14:45:29 2022] GET /identity/v3/role_assignments?user.id=67376baab7bd4cfa8a2e01f91535389e&scope.project.id=b817b0f034a64d72914672463eed8f62&role.id=869a5d710fb347a195e8f7b2cd3f0750 => generated 259 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:45:29.301720 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b843ab96-ac58-438a-8151-d73456193b26 None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:29.301901 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b843ab96-ac58-438a-8151-d73456193b26 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:29.302171 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b843ab96-ac58-438a-8151-d73456193b26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:29.336045 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-b843ab96-ac58-438a-8151-d73456193b26 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:45:29.337312 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-b843ab96-ac58-438a-8151-d73456193b26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:45:29.369541 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0bf475-ac43-4337-b4fa-c4fd66c5e029 None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:29.373449 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:29.485276 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 122/253] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 1 14:45:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 14:45:29.492195 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b843ab96-ac58-438a-8151-d73456193b26 None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:29.493377 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1b4481-9579-474e-9139-68a0e548aa4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:29.494026 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1b4481-9579-474e-9139-68a0e548aa4f None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:29.494330 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1b4481-9579-474e-9139-68a0e548aa4f None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:29.494631 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1b4481-9579-474e-9139-68a0e548aa4f None admin] PATH_INFO: `/v3/roles` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:29.495325 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1b4481-9579-474e-9139-68a0e548aa4f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:45:29.495627 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1b4481-9579-474e-9139-68a0e548aa4f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:29.497023 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1b4481-9579-474e-9139-68a0e548aa4f None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:29.525861 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 132/254] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Sep 1 14:45:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:45:29.531765 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1b4481-9579-474e-9139-68a0e548aa4f None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:29.622215 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b843ab96-ac58-438a-8151-d73456193b26 None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:29.664409 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadb0147-9750-47ae-8b55-5758ce8f9320 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:29.665056 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadb0147-9750-47ae-8b55-5758ce8f9320 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:29.665340 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadb0147-9750-47ae-8b55-5758ce8f9320 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:29.665596 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadb0147-9750-47ae-8b55-5758ce8f9320 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:29.676693 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eadb0147-9750-47ae-8b55-5758ce8f9320 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 14:45:29.682165 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application [None req-eadb0147-9750-47ae-8b55-5758ce8f9320 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 14:45:29.682165 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:45:29.682165 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:45:29.682165 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:45:29.682165 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:45:29.682165 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:45:29.682165 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:45:29.682165 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:45:29.682165 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:45:29.682165 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:45:29.682165 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:45:29.682165 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:45:29.682165 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 14:45:29.682165 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 14:45:29.682165 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 14:45:29.682165 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 14:45:29.683115 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 14:45:29.683115 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 14:45:29.683115 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 14:45:29.683115 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 14:45:29.683115 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:45:29.683115 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:45:29.683115 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:45:29.683115 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:45:29.683115 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:45:29.683115 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:45:29.683115 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:45:29.683115 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application with Lock( Sep 01 14:45:29.683115 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:45:29.683115 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:45:29.683115 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:45:29.683115 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:45:29.683920 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:45:29.683920 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:45:29.683920 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:45:29.683920 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:45:29.683920 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 14:45:29.683920 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 14:45:29.683920 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 14:45:29.683920 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 14:45:29.683920 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 14:45:29.683920 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 14:45:29.683920 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 14:45:29.683920 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Sep 01 14:45:29.683920 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 123/255] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Sep 1 14:45:29 2022] GET /identity/v3/roles/member => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 14:45:29.691140 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadb0147-9750-47ae-8b55-5758ce8f9320 None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:29.719246 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:29.727347 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:29.769381 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1b4481-9579-474e-9139-68a0e548aa4f None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:29.833018 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c99800c-e72f-4b3d-8781-138a4fd7bcae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:29.834049 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c99800c-e72f-4b3d-8781-138a4fd7bcae None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:29.835319 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c99800c-e72f-4b3d-8781-138a4fd7bcae None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:29.838446 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c99800c-e72f-4b3d-8781-138a4fd7bcae None admin] PATH_INFO: `/v3/users/80eb7b456bec41828b3adc680eabbadb` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:29.839902 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c99800c-e72f-4b3d-8781-138a4fd7bcae None admin] RBAC: Authorizing `identity:get_user(user_id=80eb7b456bec41828b3adc680eabbadb)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:29.841598 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c99800c-e72f-4b3d-8781-138a4fd7bcae None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:29.843999 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 133/256] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Sep 1 14:45:29 2022] GET /identity/v3/users/80eb7b456bec41828b3adc680eabbadb => generated 283 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:45:29.852380 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c99800c-e72f-4b3d-8781-138a4fd7bcae None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:29.877111 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadb0147-9750-47ae-8b55-5758ce8f9320 None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:29.951372 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9819691-84eb-48d2-8d07-e56a7b0a6d77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:29.952105 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9819691-84eb-48d2-8d07-e56a7b0a6d77 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:29.952377 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9819691-84eb-48d2-8d07-e56a7b0a6d77 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:29.952653 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9819691-84eb-48d2-8d07-e56a7b0a6d77 None admin] PATH_INFO: `/v3/roles` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:29.953356 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9819691-84eb-48d2-8d07-e56a7b0a6d77 None admin] RBAC: Adding query filter params (name=member) {{(pid=81341) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:45:29.953632 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9819691-84eb-48d2-8d07-e56a7b0a6d77 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:29.955150 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9819691-84eb-48d2-8d07-e56a7b0a6d77 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:29.978120 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 124/257] 10.222.0.33 () {56 vars in 1101 bytes} [Thu Sep 1 14:45:29 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 14:45:29.981476 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9819691-84eb-48d2-8d07-e56a7b0a6d77 None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:29.997736 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c99800c-e72f-4b3d-8781-138a4fd7bcae None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:30.032847 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85ec6d1-e705-4fbb-92a0-fe1823168a27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:30.033815 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85ec6d1-e705-4fbb-92a0-fe1823168a27 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:30.034316 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85ec6d1-e705-4fbb-92a0-fe1823168a27 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:30.034802 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85ec6d1-e705-4fbb-92a0-fe1823168a27 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:30.062639 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:30.117896 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85ec6d1-e705-4fbb-92a0-fe1823168a27 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:30.119851 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85ec6d1-e705-4fbb-92a0-fe1823168a27 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:30.154150 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:30.219646 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9819691-84eb-48d2-8d07-e56a7b0a6d77 None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:30.230912 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 134/258] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Sep 1 14:45:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 14:45:30.232185 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85ec6d1-e705-4fbb-92a0-fe1823168a27 None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:30.281293 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:30.291604 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8432339d-6e9a-4cb7-ad67-50d2ef318163 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:30.292192 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8432339d-6e9a-4cb7-ad67-50d2ef318163 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:30.292478 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8432339d-6e9a-4cb7-ad67-50d2ef318163 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:30.292751 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8432339d-6e9a-4cb7-ad67-50d2ef318163 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:30.314297 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8432339d-6e9a-4cb7-ad67-50d2ef318163 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 14:45:30.314297 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application [None req-8432339d-6e9a-4cb7-ad67-50d2ef318163 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 14:45:30.314297 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:45:30.314297 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:45:30.314297 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:45:30.314297 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:45:30.314297 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:45:30.314297 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:45:30.314297 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:45:30.314297 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:45:30.314297 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:45:30.314297 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:45:30.314297 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:45:30.314297 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 14:45:30.315161 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 14:45:30.315161 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 14:45:30.315161 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 14:45:30.315161 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 14:45:30.315161 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 14:45:30.315161 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 14:45:30.315161 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 14:45:30.315161 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:45:30.315161 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:45:30.315161 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:45:30.315161 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:45:30.315161 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:45:30.315161 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:45:30.315161 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:45:30.315161 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application with Lock( Sep 01 14:45:30.315161 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:45:30.315161 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:45:30.316043 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:45:30.316043 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:45:30.316043 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:45:30.316043 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:45:30.316043 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:45:30.316043 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:45:30.316043 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 14:45:30.316043 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 14:45:30.316043 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 14:45:30.316043 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 14:45:30.316043 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 14:45:30.316043 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 14:45:30.316043 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 14:45:30.316043 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Sep 01 14:45:30.316043 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 125/259] 10.222.0.33 () {56 vars in 1114 bytes} [Thu Sep 1 14:45:29 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 337 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:45:30.316901 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8432339d-6e9a-4cb7-ad67-50d2ef318163 None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:30.379906 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85ec6d1-e705-4fbb-92a0-fe1823168a27 None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:30.426305 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8432339d-6e9a-4cb7-ad67-50d2ef318163 None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:30.447771 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88c610c-7ad0-4bbf-a988-1967f527ad40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:30.448666 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88c610c-7ad0-4bbf-a988-1967f527ad40 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:30.449071 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:30.450174 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88c610c-7ad0-4bbf-a988-1967f527ad40 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:30.450635 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88c610c-7ad0-4bbf-a988-1967f527ad40 None admin] PATH_INFO: `/v3/users/ea5aa8fb75dc489ea6e3bdbcd21263a5` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:30.451904 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88c610c-7ad0-4bbf-a988-1967f527ad40 None admin] RBAC: Authorizing `identity:get_user(user_id=ea5aa8fb75dc489ea6e3bdbcd21263a5)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:30.453488 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88c610c-7ad0-4bbf-a988-1967f527ad40 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:30.458274 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 135/260] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Sep 1 14:45:30 2022] GET /identity/v3/users/ea5aa8fb75dc489ea6e3bdbcd21263a5 => generated 297 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:45:30.459964 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88c610c-7ad0-4bbf-a988-1967f527ad40 None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:30.462001 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750b9d91-4ccd-4b06-8d04-b23b34040290 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:30.463269 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750b9d91-4ccd-4b06-8d04-b23b34040290 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:30.463909 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750b9d91-4ccd-4b06-8d04-b23b34040290 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:30.464401 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750b9d91-4ccd-4b06-8d04-b23b34040290 None admin] PATH_INFO: `/v3/projects/b817b0f034a64d72914672463eed8f62` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:30.465751 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750b9d91-4ccd-4b06-8d04-b23b34040290 None admin] RBAC: Authorizing `identity:get_project(project_id=b817b0f034a64d72914672463eed8f62)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:30.467675 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750b9d91-4ccd-4b06-8d04-b23b34040290 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:30.469511 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 126/261] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Sep 1 14:45:30 2022] GET /identity/v3/projects/b817b0f034a64d72914672463eed8f62 => generated 302 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:45:30.470677 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750b9d91-4ccd-4b06-8d04-b23b34040290 None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:30.575619 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750b9d91-4ccd-4b06-8d04-b23b34040290 None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:30.611932 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7160805d-6991-41a7-b21f-e117f3e3e332 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:30.612943 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7160805d-6991-41a7-b21f-e117f3e3e332 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:30.613472 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7160805d-6991-41a7-b21f-e117f3e3e332 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:30.613969 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7160805d-6991-41a7-b21f-e117f3e3e332 None admin] PATH_INFO: `/v3/projects/b817b0f034a64d72914672463eed8f62` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:30.615402 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7160805d-6991-41a7-b21f-e117f3e3e332 None admin] RBAC: Authorizing `identity:get_project(project_id=b817b0f034a64d72914672463eed8f62)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:30.617077 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7160805d-6991-41a7-b21f-e117f3e3e332 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:30.619134 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 127/262] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Sep 1 14:45:30 2022] GET /identity/v3/projects/b817b0f034a64d72914672463eed8f62 => generated 302 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:45:30.620119 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7160805d-6991-41a7-b21f-e117f3e3e332 None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:30.663690 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88c610c-7ad0-4bbf-a988-1967f527ad40 None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:30.686725 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:30.691977 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:30.738161 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0f7721-5d16-4423-8972-634e51def03e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:30.739543 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0f7721-5d16-4423-8972-634e51def03e None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:30.741537 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0f7721-5d16-4423-8972-634e51def03e None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:30.742016 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0f7721-5d16-4423-8972-634e51def03e None admin] PATH_INFO: `/v3/roles` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:30.742931 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0f7721-5d16-4423-8972-634e51def03e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:45:30.744080 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0f7721-5d16-4423-8972-634e51def03e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:30.747583 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0f7721-5d16-4423-8972-634e51def03e None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:30.749294 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7160805d-6991-41a7-b21f-e117f3e3e332 None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:30.766940 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 136/263] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Sep 1 14:45:30 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:45:30.768069 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0f7721-5d16-4423-8972-634e51def03e None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:30.814985 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bf1cde-6f04-4b1d-88c4-a135290fd627 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:30.815967 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bf1cde-6f04-4b1d-88c4-a135290fd627 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:30.816470 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bf1cde-6f04-4b1d-88c4-a135290fd627 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:30.816960 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bf1cde-6f04-4b1d-88c4-a135290fd627 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:30.817889 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bf1cde-6f04-4b1d-88c4-a135290fd627 None admin] RBAC: Adding query filter params (role.id=2a2adb769e55476b82e5014897e0a9c0, scope.project.id=b817b0f034a64d72914672463eed8f62, user.id=80eb7b456bec41828b3adc680eabbadb) {{(pid=81341) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:45:30.819053 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bf1cde-6f04-4b1d-88c4-a135290fd627 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:30.820910 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bf1cde-6f04-4b1d-88c4-a135290fd627 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:30.827793 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 128/264] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Sep 1 14:45:30 2022] GET /identity/v3/role_assignments?user.id=80eb7b456bec41828b3adc680eabbadb&scope.project.id=b817b0f034a64d72914672463eed8f62&role.id=2a2adb769e55476b82e5014897e0a9c0 => generated 608 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:45:30.828865 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bf1cde-6f04-4b1d-88c4-a135290fd627 None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:30.944832 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bf1cde-6f04-4b1d-88c4-a135290fd627 None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:30.972695 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0f7721-5d16-4423-8972-634e51def03e None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:30.978474 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:30.994016 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f3505c-28b7-4db7-ae9c-4c8749329070 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:30.994697 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f3505c-28b7-4db7-ae9c-4c8749329070 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:30.994986 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f3505c-28b7-4db7-ae9c-4c8749329070 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:30.995267 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f3505c-28b7-4db7-ae9c-4c8749329070 None admin] PATH_INFO: `/v3/users/80eb7b456bec41828b3adc680eabbadb` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:30.998484 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f3505c-28b7-4db7-ae9c-4c8749329070 None admin] RBAC: Authorizing `identity:get_user(user_id=80eb7b456bec41828b3adc680eabbadb)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:31.002052 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f3505c-28b7-4db7-ae9c-4c8749329070 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:31.007069 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 129/265] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Sep 1 14:45:30 2022] GET /identity/v3/users/80eb7b456bec41828b3adc680eabbadb => generated 283 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:45:31.012458 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f3505c-28b7-4db7-ae9c-4c8749329070 None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:31.044442 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf35334a-1a3d-4669-9a42-6259a5c16891 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:31.045492 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf35334a-1a3d-4669-9a42-6259a5c16891 None admin] REQUEST_METHOD: `PUT` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:31.045988 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf35334a-1a3d-4669-9a42-6259a5c16891 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:31.046492 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf35334a-1a3d-4669-9a42-6259a5c16891 None admin] PATH_INFO: `/v3/projects/b817b0f034a64d72914672463eed8f62/users/ea5aa8fb75dc489ea6e3bdbcd21263a5/roles/5d1feef0e13d4874928bd86258c4e3b2` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:31.058005 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf35334a-1a3d-4669-9a42-6259a5c16891 None admin] RBAC: Authorizing `identity:create_grant(project_id=b817b0f034a64d72914672463eed8f62, user_id=ea5aa8fb75dc489ea6e3bdbcd21263a5, role_id=5d1feef0e13d4874928bd86258c4e3b2)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:31.059816 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf35334a-1a3d-4669-9a42-6259a5c16891 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:31.117138 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 137/266] 10.222.0.33 () {58 vars in 1439 bytes} [Thu Sep 1 14:45:30 2022] PUT /identity/v3/projects/b817b0f034a64d72914672463eed8f62/users/ea5aa8fb75dc489ea6e3bdbcd21263a5/roles/5d1feef0e13d4874928bd86258c4e3b2 => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:45:31.201902 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f3505c-28b7-4db7-ae9c-4c8749329070 None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:31.242523 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d41d60-4a19-4d35-8987-21be57ea7407 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:31.243159 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d41d60-4a19-4d35-8987-21be57ea7407 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:31.243440 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d41d60-4a19-4d35-8987-21be57ea7407 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:31.243712 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d41d60-4a19-4d35-8987-21be57ea7407 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:31.318601 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:31.327487 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d41d60-4a19-4d35-8987-21be57ea7407 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:31.328998 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d41d60-4a19-4d35-8987-21be57ea7407 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:31.441976 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 130/267] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Sep 1 14:45:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 14:45:31.448346 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf35334a-1a3d-4669-9a42-6259a5c16891 None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:31.487625 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:31.531250 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f96452-3be1-4191-a971-3d0f06eb5133 None None] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:31.531602 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f96452-3be1-4191-a971-3d0f06eb5133 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:31.531869 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f96452-3be1-4191-a971-3d0f06eb5133 None None] PATH_INFO: `/` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:31.533471 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 131/268] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 1 14:45:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:45:31.539761 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322cb736-7a8b-4fec-8581-a0a184a8cf4c None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:31.540037 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322cb736-7a8b-4fec-8581-a0a184a8cf4c None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:31.540304 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322cb736-7a8b-4fec-8581-a0a184a8cf4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:31.567825 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-322cb736-7a8b-4fec-8581-a0a184a8cf4c None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:45:31.569053 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-322cb736-7a8b-4fec-8581-a0a184a8cf4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:45:31.600615 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf35334a-1a3d-4669-9a42-6259a5c16891 None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:31.668630 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04396bcd-1963-4df4-8068-d3d0e2bfdfc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:31.669261 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04396bcd-1963-4df4-8068-d3d0e2bfdfc6 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:31.669586 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04396bcd-1963-4df4-8068-d3d0e2bfdfc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:31.669854 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04396bcd-1963-4df4-8068-d3d0e2bfdfc6 None admin] PATH_INFO: `/v3/projects/b817b0f034a64d72914672463eed8f62` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:31.678770 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04396bcd-1963-4df4-8068-d3d0e2bfdfc6 None admin] RBAC: Authorizing `identity:get_project(project_id=b817b0f034a64d72914672463eed8f62)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:31.680265 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04396bcd-1963-4df4-8068-d3d0e2bfdfc6 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:31.682387 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 138/269] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Sep 1 14:45:31 2022] GET /identity/v3/projects/b817b0f034a64d72914672463eed8f62 => generated 302 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:45:31.688153 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04396bcd-1963-4df4-8068-d3d0e2bfdfc6 None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:31.689389 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 132/270] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 1 14:45:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 14:45:31.712904 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f1aecb-aeda-404a-b74d-a3e784e48f8f None None] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:31.713803 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f1aecb-aeda-404a-b74d-a3e784e48f8f None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:31.714216 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f1aecb-aeda-404a-b74d-a3e784e48f8f None None] PATH_INFO: `/` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:31.716236 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 133/271] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 1 14:45:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 14:45:31.720962 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470dfafd-fd69-4d39-ba8e-af0d7a15b237 None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:31.721414 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470dfafd-fd69-4d39-ba8e-af0d7a15b237 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:31.721836 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470dfafd-fd69-4d39-ba8e-af0d7a15b237 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:31.759298 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-470dfafd-fd69-4d39-ba8e-af0d7a15b237 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:45:31.760983 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-470dfafd-fd69-4d39-ba8e-af0d7a15b237 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:45:31.794507 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:31.804115 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04396bcd-1963-4df4-8068-d3d0e2bfdfc6 None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:31.811400 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:31.841754 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a9d756-c33e-49fc-b711-f39473ac9bea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:31.842434 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a9d756-c33e-49fc-b711-f39473ac9bea None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:31.842727 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a9d756-c33e-49fc-b711-f39473ac9bea None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:31.842999 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a9d756-c33e-49fc-b711-f39473ac9bea None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:31.843747 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a9d756-c33e-49fc-b711-f39473ac9bea None admin] RBAC: Adding query filter params (role.id=9996407d588d4a00836c8c50e7f37b1a, scope.project.id=b817b0f034a64d72914672463eed8f62, user.id=80eb7b456bec41828b3adc680eabbadb) {{(pid=81342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:45:31.846262 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a9d756-c33e-49fc-b711-f39473ac9bea None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:31.846262 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a9d756-c33e-49fc-b711-f39473ac9bea None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:31.858413 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 139/272] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Sep 1 14:45:31 2022] GET /identity/v3/role_assignments?user.id=80eb7b456bec41828b3adc680eabbadb&scope.project.id=b817b0f034a64d72914672463eed8f62&role.id=9996407d588d4a00836c8c50e7f37b1a => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 14:45:31.865122 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c947f5f-5ab5-44e9-85e9-c43f41c71748 None None] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:31.865455 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c947f5f-5ab5-44e9-85e9-c43f41c71748 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:31.865732 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c947f5f-5ab5-44e9-85e9-c43f41c71748 None None] PATH_INFO: `/` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:31.867381 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 140/273] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 1 14:45:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 14:45:31.876099 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc874a89-9f12-456d-b91f-36c055fe0dd2 None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:31.876383 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc874a89-9f12-456d-b91f-36c055fe0dd2 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:31.876658 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc874a89-9f12-456d-b91f-36c055fe0dd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:31.884364 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 134/274] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 1 14:45:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 14:45:31.901486 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-dc874a89-9f12-456d-b91f-36c055fe0dd2 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:45:31.903029 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-dc874a89-9f12-456d-b91f-36c055fe0dd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:45:31.991675 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 141/275] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 1 14:45:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 14:45:31.997960 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470dfafd-fd69-4d39-ba8e-af0d7a15b237 None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:32.129664 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470dfafd-fd69-4d39-ba8e-af0d7a15b237 None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:32.170788 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c6922c-5c26-4098-9626-9b585a9bef54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:32.171811 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c6922c-5c26-4098-9626-9b585a9bef54 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:32.172283 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c6922c-5c26-4098-9626-9b585a9bef54 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:32.172766 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c6922c-5c26-4098-9626-9b585a9bef54 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:32.183315 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c6922c-5c26-4098-9626-9b585a9bef54 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 14:45:32.188267 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application [None req-75c6922c-5c26-4098-9626-9b585a9bef54 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 14:45:32.188267 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:45:32.188267 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:45:32.188267 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:45:32.188267 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:45:32.188267 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:45:32.188267 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:45:32.188267 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:45:32.188267 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:45:32.188267 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:45:32.188267 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:45:32.188267 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:45:32.188267 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 14:45:32.188267 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 14:45:32.188267 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 14:45:32.188267 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 14:45:32.189177 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 14:45:32.189177 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 14:45:32.189177 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 14:45:32.189177 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 14:45:32.189177 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:45:32.189177 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:45:32.189177 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:45:32.189177 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:45:32.189177 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:45:32.189177 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:45:32.189177 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:45:32.189177 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application with Lock( Sep 01 14:45:32.189177 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:45:32.189177 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:45:32.189177 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:45:32.189177 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:45:32.189989 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:45:32.189989 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:45:32.189989 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:45:32.189989 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:45:32.189989 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 14:45:32.189989 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 14:45:32.189989 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 14:45:32.189989 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 14:45:32.189989 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 14:45:32.189989 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 14:45:32.189989 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 14:45:32.189989 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Sep 01 14:45:32.192307 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 135/276] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Sep 1 14:45:31 2022] GET /identity/v3/roles/member => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:45:32.198973 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc874a89-9f12-456d-b91f-36c055fe0dd2 None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:32.256634 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c6922c-5c26-4098-9626-9b585a9bef54 None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:32.282038 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:32.331781 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:32.334216 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc874a89-9f12-456d-b91f-36c055fe0dd2 None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:32.373247 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c9f0dd-9574-4407-8409-f4dfcaa42fe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:32.373855 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c9f0dd-9574-4407-8409-f4dfcaa42fe2 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:32.374447 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c9f0dd-9574-4407-8409-f4dfcaa42fe2 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:32.374447 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c9f0dd-9574-4407-8409-f4dfcaa42fe2 None admin] PATH_INFO: `/v3/roles` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:32.375091 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c9f0dd-9574-4407-8409-f4dfcaa42fe2 None admin] RBAC: Adding query filter params (name=member) {{(pid=81342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:45:32.375399 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c9f0dd-9574-4407-8409-f4dfcaa42fe2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:32.376781 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c9f0dd-9574-4407-8409-f4dfcaa42fe2 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:32.389878 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 142/277] 10.222.0.33 () {56 vars in 1101 bytes} [Thu Sep 1 14:45:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 14:45:32.389878 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c9f0dd-9574-4407-8409-f4dfcaa42fe2 None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:32.392640 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c6922c-5c26-4098-9626-9b585a9bef54 None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:32.427043 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa752440-1ca3-4d54-97f6-7f6fbc9a2920 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:32.428175 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa752440-1ca3-4d54-97f6-7f6fbc9a2920 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:32.428452 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa752440-1ca3-4d54-97f6-7f6fbc9a2920 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:32.428714 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa752440-1ca3-4d54-97f6-7f6fbc9a2920 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:32.429808 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa752440-1ca3-4d54-97f6-7f6fbc9a2920 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:32.431322 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa752440-1ca3-4d54-97f6-7f6fbc9a2920 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:32.432873 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 136/278] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 1 14:45:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:45:32.433614 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa752440-1ca3-4d54-97f6-7f6fbc9a2920 None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:32.500171 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c9f0dd-9574-4407-8409-f4dfcaa42fe2 None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:32.524167 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:32.538872 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4903fb-73df-4642-a777-7df1ae3ab04d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:32.539291 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa752440-1ca3-4d54-97f6-7f6fbc9a2920 None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:32.539506 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4903fb-73df-4642-a777-7df1ae3ab04d None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:32.539786 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4903fb-73df-4642-a777-7df1ae3ab04d None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:32.540055 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4903fb-73df-4642-a777-7df1ae3ab04d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:32.551218 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4903fb-73df-4642-a777-7df1ae3ab04d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 14:45:32.557624 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application [None req-4f4903fb-73df-4642-a777-7df1ae3ab04d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 14:45:32.557624 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:45:32.557624 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:45:32.557624 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:45:32.557624 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:45:32.557624 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:45:32.557624 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:45:32.557624 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:45:32.557624 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:45:32.557624 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:45:32.557624 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:45:32.557624 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:45:32.557624 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 14:45:32.557624 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 14:45:32.557624 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 14:45:32.557624 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 14:45:32.558782 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 14:45:32.558782 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 14:45:32.558782 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 14:45:32.558782 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 14:45:32.558782 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:45:32.558782 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:45:32.558782 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:45:32.558782 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:45:32.558782 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:45:32.558782 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:45:32.558782 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:45:32.558782 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application with Lock( Sep 01 14:45:32.558782 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:45:32.558782 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:45:32.558782 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:45:32.558782 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:45:32.559608 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:45:32.559608 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:45:32.559608 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:45:32.559608 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:45:32.559608 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 14:45:32.559608 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 14:45:32.559608 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 14:45:32.559608 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 14:45:32.559608 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 14:45:32.559608 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 14:45:32.559608 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 14:45:32.559608 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Sep 01 14:45:32.559608 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 143/279] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Sep 1 14:45:32 2022] GET /identity/v3/roles/reader => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:45:32.560285 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4903fb-73df-4642-a777-7df1ae3ab04d None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:32.576694 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2901893-3fda-4215-8ed3-7d330178db16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:32.577416 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2901893-3fda-4215-8ed3-7d330178db16 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:32.577753 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2901893-3fda-4215-8ed3-7d330178db16 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:32.578086 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2901893-3fda-4215-8ed3-7d330178db16 None admin] PATH_INFO: `/v3/users/0880b1f90dc44ff183565a21fec8f8a0` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:32.586200 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:32.594586 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2901893-3fda-4215-8ed3-7d330178db16 None admin] RBAC: Authorizing `identity:get_user(user_id=0880b1f90dc44ff183565a21fec8f8a0)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:32.596278 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2901893-3fda-4215-8ed3-7d330178db16 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:32.598197 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 137/280] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Sep 1 14:45:32 2022] GET /identity/v3/users/0880b1f90dc44ff183565a21fec8f8a0 => generated 293 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:45:32.598966 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2901893-3fda-4215-8ed3-7d330178db16 None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:32.674638 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4903fb-73df-4642-a777-7df1ae3ab04d None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:32.709401 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8228557c-d7af-4090-aab6-9272495d32a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:32.710009 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8228557c-d7af-4090-aab6-9272495d32a4 None admin] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:32.710318 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8228557c-d7af-4090-aab6-9272495d32a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:32.710596 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8228557c-d7af-4090-aab6-9272495d32a4 None admin] PATH_INFO: `/v3/users` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:32.711247 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8228557c-d7af-4090-aab6-9272495d32a4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:32.712703 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8228557c-d7af-4090-aab6-9272495d32a4 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:32.721413 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2901893-3fda-4215-8ed3-7d330178db16 None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:32.735050 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 144/281] 10.222.0.33 () {60 vars in 1132 bytes} [Thu Sep 1 14:45:32 2022] POST /identity/v3/users => generated 293 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:45:32.739834 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8228557c-d7af-4090-aab6-9272495d32a4 None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:32.763265 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a98fb4d-4d2d-4715-a2ac-d478fc824c2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:32.764236 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a98fb4d-4d2d-4715-a2ac-d478fc824c2a None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:32.764742 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a98fb4d-4d2d-4715-a2ac-d478fc824c2a None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:32.765254 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a98fb4d-4d2d-4715-a2ac-d478fc824c2a None admin] PATH_INFO: `/v3/roles` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:32.766260 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a98fb4d-4d2d-4715-a2ac-d478fc824c2a None admin] RBAC: Adding query filter params (name=reader) {{(pid=81341) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:45:32.766774 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a98fb4d-4d2d-4715-a2ac-d478fc824c2a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:32.768578 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a98fb4d-4d2d-4715-a2ac-d478fc824c2a None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:32.779280 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 138/282] 10.222.0.33 () {56 vars in 1101 bytes} [Thu Sep 1 14:45:32 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:45:32.785096 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a98fb4d-4d2d-4715-a2ac-d478fc824c2a None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:32.793063 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:32.847322 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:32.878654 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8228557c-d7af-4090-aab6-9272495d32a4 None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:32.925464 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392ffc8b-0e48-419f-8e02-c5893d92a2f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:32.926108 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392ffc8b-0e48-419f-8e02-c5893d92a2f1 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:32.926396 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392ffc8b-0e48-419f-8e02-c5893d92a2f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:32.926670 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392ffc8b-0e48-419f-8e02-c5893d92a2f1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:32.927410 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392ffc8b-0e48-419f-8e02-c5893d92a2f1 None admin] RBAC: Adding query filter params (role.id=5d1feef0e13d4874928bd86258c4e3b2, user.id=0880b1f90dc44ff183565a21fec8f8a0, scope.system=all) {{(pid=81342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:45:32.928271 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392ffc8b-0e48-419f-8e02-c5893d92a2f1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:32.929738 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392ffc8b-0e48-419f-8e02-c5893d92a2f1 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:32.933380 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a98fb4d-4d2d-4715-a2ac-d478fc824c2a None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:32.942268 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 145/283] 10.222.0.33 () {56 vars in 1308 bytes} [Thu Sep 1 14:45:32 2022] GET /identity/v3/role_assignments?user.id=0880b1f90dc44ff183565a21fec8f8a0&scope.system=all&role.id=5d1feef0e13d4874928bd86258c4e3b2 => generated 226 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:45:32.981273 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5ea0bb-d4d4-4e62-bbc5-0053345d9dcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:32.981929 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5ea0bb-d4d4-4e62-bbc5-0053345d9dcf None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:32.982211 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5ea0bb-d4d4-4e62-bbc5-0053345d9dcf None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:32.982520 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5ea0bb-d4d4-4e62-bbc5-0053345d9dcf None admin] PATH_INFO: `/v3/users/67376baab7bd4cfa8a2e01f91535389e` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:32.983735 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5ea0bb-d4d4-4e62-bbc5-0053345d9dcf None admin] RBAC: Authorizing `identity:get_user(user_id=67376baab7bd4cfa8a2e01f91535389e)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:32.985263 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5ea0bb-d4d4-4e62-bbc5-0053345d9dcf None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:32.987201 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 139/284] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Sep 1 14:45:32 2022] GET /identity/v3/users/67376baab7bd4cfa8a2e01f91535389e => generated 297 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:45:32.993315 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392ffc8b-0e48-419f-8e02-c5893d92a2f1 None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:33.115757 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392ffc8b-0e48-419f-8e02-c5893d92a2f1 None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:33.153441 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5f05ab-7ae0-43e4-9dba-3e81243dcd05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:33.154344 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5f05ab-7ae0-43e4-9dba-3e81243dcd05 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:33.154601 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5f05ab-7ae0-43e4-9dba-3e81243dcd05 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:33.154870 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5f05ab-7ae0-43e4-9dba-3e81243dcd05 None admin] PATH_INFO: `/v3/projects/b817b0f034a64d72914672463eed8f62` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:33.156010 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5f05ab-7ae0-43e4-9dba-3e81243dcd05 None admin] RBAC: Authorizing `identity:get_project(project_id=b817b0f034a64d72914672463eed8f62)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:33.157460 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5f05ab-7ae0-43e4-9dba-3e81243dcd05 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:33.159418 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 146/285] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Sep 1 14:45:32 2022] GET /identity/v3/projects/b817b0f034a64d72914672463eed8f62 => generated 302 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:45:33.165662 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5ea0bb-d4d4-4e62-bbc5-0053345d9dcf None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:33.242083 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:33.275725 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5ea0bb-d4d4-4e62-bbc5-0053345d9dcf None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:33.317832 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0630aaa2-fd08-4517-852d-f74ed6ef3e3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:33.319347 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0630aaa2-fd08-4517-852d-f74ed6ef3e3f None admin] REQUEST_METHOD: `PUT` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:33.319628 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0630aaa2-fd08-4517-852d-f74ed6ef3e3f None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:33.319904 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0630aaa2-fd08-4517-852d-f74ed6ef3e3f None admin] PATH_INFO: `/v3/projects/b817b0f034a64d72914672463eed8f62/users/67376baab7bd4cfa8a2e01f91535389e/roles/869a5d710fb347a195e8f7b2cd3f0750` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:33.327027 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0630aaa2-fd08-4517-852d-f74ed6ef3e3f None admin] RBAC: Authorizing `identity:create_grant(project_id=b817b0f034a64d72914672463eed8f62, user_id=67376baab7bd4cfa8a2e01f91535389e, role_id=869a5d710fb347a195e8f7b2cd3f0750)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:33.328542 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0630aaa2-fd08-4517-852d-f74ed6ef3e3f None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:33.358184 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 140/286] 10.222.0.33 () {58 vars in 1439 bytes} [Thu Sep 1 14:45:33 2022] PUT /identity/v3/projects/b817b0f034a64d72914672463eed8f62/users/67376baab7bd4cfa8a2e01f91535389e/roles/869a5d710fb347a195e8f7b2cd3f0750 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:45:33.493468 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf61f9a8-47b9-4cf0-b1ff-416357449df6 None None] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:33.493655 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf61f9a8-47b9-4cf0-b1ff-416357449df6 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:33.493911 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf61f9a8-47b9-4cf0-b1ff-416357449df6 None None] PATH_INFO: `/` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:33.495306 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 147/287] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 1 14:45:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:45:33.502944 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d84cad-9b60-47a3-a027-dd4dd5a4887d None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:33.503251 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d84cad-9b60-47a3-a027-dd4dd5a4887d None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:33.503500 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d84cad-9b60-47a3-a027-dd4dd5a4887d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:33.536036 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-a4d84cad-9b60-47a3-a027-dd4dd5a4887d None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:45:33.537303 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-a4d84cad-9b60-47a3-a027-dd4dd5a4887d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:45:33.620191 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 141/288] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 1 14:45:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 14:45:33.929481 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf61f9a8-47b9-4cf0-b1ff-416357449df6 None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:33.942113 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:34.056549 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf61f9a8-47b9-4cf0-b1ff-416357449df6 None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:34.098163 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc62fb5-8d68-4157-a295-ac71060066ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:34.098825 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc62fb5-8d68-4157-a295-ac71060066ee None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:34.099102 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc62fb5-8d68-4157-a295-ac71060066ee None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:34.099369 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc62fb5-8d68-4157-a295-ac71060066ee None admin] PATH_INFO: `/v3/roles/member` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:34.115904 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc62fb5-8d68-4157-a295-ac71060066ee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 14:45:34.120766 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application [None req-bfc62fb5-8d68-4157-a295-ac71060066ee None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 14:45:34.120766 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:45:34.120766 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:45:34.120766 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:45:34.120766 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:45:34.120766 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:45:34.120766 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:45:34.120766 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:45:34.120766 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:45:34.120766 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:45:34.120766 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:45:34.120766 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:45:34.120766 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 14:45:34.120766 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 14:45:34.120766 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 14:45:34.120766 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 14:45:34.121676 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 14:45:34.121676 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 14:45:34.121676 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 14:45:34.121676 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 14:45:34.121676 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:45:34.121676 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:45:34.121676 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:45:34.121676 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:45:34.121676 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:45:34.121676 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:45:34.121676 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:45:34.121676 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application with Lock( Sep 01 14:45:34.121676 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:45:34.121676 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:45:34.121676 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:45:34.121676 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:45:34.122513 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:45:34.122513 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:45:34.122513 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:45:34.122513 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:45:34.122513 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 14:45:34.122513 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 14:45:34.122513 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 14:45:34.122513 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 14:45:34.122513 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 14:45:34.122513 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 14:45:34.122513 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 14:45:34.122513 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Sep 01 14:45:34.122513 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 148/289] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Sep 1 14:45:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:45:34.128104 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d84cad-9b60-47a3-a027-dd4dd5a4887d None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:34.207913 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:34.254223 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d84cad-9b60-47a3-a027-dd4dd5a4887d None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:34.300470 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6791c85-8761-4614-b491-80ea7dfea6cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:34.301076 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6791c85-8761-4614-b491-80ea7dfea6cd None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:34.301349 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6791c85-8761-4614-b491-80ea7dfea6cd None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:34.301614 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6791c85-8761-4614-b491-80ea7dfea6cd None admin] PATH_INFO: `/v3/roles` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:34.302290 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6791c85-8761-4614-b491-80ea7dfea6cd None admin] RBAC: Adding query filter params (name=member) {{(pid=81341) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:45:34.302614 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6791c85-8761-4614-b491-80ea7dfea6cd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:34.304513 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6791c85-8761-4614-b491-80ea7dfea6cd None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:34.317367 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 142/290] 10.222.0.33 () {56 vars in 1101 bytes} [Thu Sep 1 14:45:34 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:45:34.323182 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc62fb5-8d68-4157-a295-ac71060066ee None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:34.399334 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:34.436661 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc62fb5-8d68-4157-a295-ac71060066ee None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:34.478445 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab9ee59-a80e-43ca-9261-21bb088dde49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:34.478996 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab9ee59-a80e-43ca-9261-21bb088dde49 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:34.479273 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab9ee59-a80e-43ca-9261-21bb088dde49 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:34.479598 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab9ee59-a80e-43ca-9261-21bb088dde49 None admin] PATH_INFO: `/v3/users/ea5aa8fb75dc489ea6e3bdbcd21263a5` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:34.480768 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab9ee59-a80e-43ca-9261-21bb088dde49 None admin] RBAC: Authorizing `identity:get_user(user_id=ea5aa8fb75dc489ea6e3bdbcd21263a5)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:34.482229 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab9ee59-a80e-43ca-9261-21bb088dde49 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:34.484081 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 149/291] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Sep 1 14:45:34 2022] GET /identity/v3/users/ea5aa8fb75dc489ea6e3bdbcd21263a5 => generated 297 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:45:34.495315 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6791c85-8761-4614-b491-80ea7dfea6cd None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:34.626105 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6791c85-8761-4614-b491-80ea7dfea6cd None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:34.661790 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:34.674378 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bff66a-be2d-469d-835c-7be946088c4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:34.675042 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bff66a-be2d-469d-835c-7be946088c4f None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:34.675318 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bff66a-be2d-469d-835c-7be946088c4f None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:34.675595 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bff66a-be2d-469d-835c-7be946088c4f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:34.765944 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bff66a-be2d-469d-835c-7be946088c4f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:34.767419 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bff66a-be2d-469d-835c-7be946088c4f None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:34.874109 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 143/292] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Sep 1 14:45:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 14:45:34.879425 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab9ee59-a80e-43ca-9261-21bb088dde49 None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:35.019140 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab9ee59-a80e-43ca-9261-21bb088dde49 None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:35.048784 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf425efb-7ca4-401c-be8d-1c378f823f36 None None] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:35.049103 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf425efb-7ca4-401c-be8d-1c378f823f36 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:35.049326 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf425efb-7ca4-401c-be8d-1c378f823f36 None None] PATH_INFO: `/` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:35.050702 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 144/293] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 1 14:45:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:45:35.055578 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc75de23-2743-49ec-b254-520e81136352 None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:35.055857 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc75de23-2743-49ec-b254-520e81136352 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:35.056123 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc75de23-2743-49ec-b254-520e81136352 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:35.080547 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acd54f3-6fef-4a45-99b4-c34fd7678459 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:35.081191 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acd54f3-6fef-4a45-99b4-c34fd7678459 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:35.081468 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acd54f3-6fef-4a45-99b4-c34fd7678459 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:35.081737 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acd54f3-6fef-4a45-99b4-c34fd7678459 None admin] PATH_INFO: `/v3/projects/b817b0f034a64d72914672463eed8f62` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:35.083113 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1acd54f3-6fef-4a45-99b4-c34fd7678459 None admin] RBAC: Authorizing `identity:get_project(project_id=b817b0f034a64d72914672463eed8f62)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:35.083525 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-bc75de23-2743-49ec-b254-520e81136352 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:45:35.085143 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1acd54f3-6fef-4a45-99b4-c34fd7678459 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:35.085727 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-bc75de23-2743-49ec-b254-520e81136352 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:45:35.087130 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 150/294] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Sep 1 14:45:34 2022] GET /identity/v3/projects/b817b0f034a64d72914672463eed8f62 => generated 302 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:45:35.092908 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acd54f3-6fef-4a45-99b4-c34fd7678459 None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:35.100483 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:35.110434 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:35.200911 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 145/295] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 1 14:45:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 14:45:35.255717 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acd54f3-6fef-4a45-99b4-c34fd7678459 None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:35.303428 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeea9f67-fc78-41ed-b100-674fb42f1d93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:35.304242 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeea9f67-fc78-41ed-b100-674fb42f1d93 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:35.304680 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeea9f67-fc78-41ed-b100-674fb42f1d93 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:35.305106 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeea9f67-fc78-41ed-b100-674fb42f1d93 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:35.305958 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeea9f67-fc78-41ed-b100-674fb42f1d93 None admin] RBAC: Adding query filter params (role.id=5d1feef0e13d4874928bd86258c4e3b2, scope.project.id=b817b0f034a64d72914672463eed8f62, user.id=ea5aa8fb75dc489ea6e3bdbcd21263a5) {{(pid=81342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:45:35.306420 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeea9f67-fc78-41ed-b100-674fb42f1d93 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:35.308028 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeea9f67-fc78-41ed-b100-674fb42f1d93 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:35.319262 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 151/296] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Sep 1 14:45:35 2022] GET /identity/v3/role_assignments?user.id=ea5aa8fb75dc489ea6e3bdbcd21263a5&scope.project.id=b817b0f034a64d72914672463eed8f62&role.id=5d1feef0e13d4874928bd86258c4e3b2 => generated 608 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 14:45:35.414858 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297de1a7-f1a4-432c-9cce-14bc76fc4370 None None] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:35.415482 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297de1a7-f1a4-432c-9cce-14bc76fc4370 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:35.415955 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297de1a7-f1a4-432c-9cce-14bc76fc4370 None None] PATH_INFO: `/` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:35.417738 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 146/297] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 1 14:45:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:45:35.423182 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd760350-54b4-4fbb-ac23-bf10a5e6dc00 None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:35.423746 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd760350-54b4-4fbb-ac23-bf10a5e6dc00 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:35.424219 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd760350-54b4-4fbb-ac23-bf10a5e6dc00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:35.444153 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-cd760350-54b4-4fbb-ac23-bf10a5e6dc00 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:45:35.445709 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-cd760350-54b4-4fbb-ac23-bf10a5e6dc00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:45:35.516303 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297de1a7-f1a4-432c-9cce-14bc76fc4370 None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:35.546637 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:35.555787 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 152/298] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 1 14:45:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 14:45:35.582069 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc162394-7bff-45ec-b99d-f10d3373a9ac None None] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:35.582394 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc162394-7bff-45ec-b99d-f10d3373a9ac None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:35.582650 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc162394-7bff-45ec-b99d-f10d3373a9ac None None] PATH_INFO: `/` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:35.584155 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 153/299] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 1 14:45:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 14:45:35.589241 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeab5786-8575-4e39-b595-f678747402ae None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:35.589756 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeab5786-8575-4e39-b595-f678747402ae None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:35.590259 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeab5786-8575-4e39-b595-f678747402ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:35.611204 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-aeab5786-8575-4e39-b595-f678747402ae None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:45:35.613171 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-aeab5786-8575-4e39-b595-f678747402ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:45:35.673169 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297de1a7-f1a4-432c-9cce-14bc76fc4370 None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:35.712506 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 154/300] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 1 14:45:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 14:45:35.716096 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a91b6eb-c6be-45b5-88e7-ba8207aaeb46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:35.716903 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a91b6eb-c6be-45b5-88e7-ba8207aaeb46 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:35.717306 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a91b6eb-c6be-45b5-88e7-ba8207aaeb46 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:35.717736 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a91b6eb-c6be-45b5-88e7-ba8207aaeb46 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:35.729794 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a91b6eb-c6be-45b5-88e7-ba8207aaeb46 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 14:45:35.735090 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application [None req-4a91b6eb-c6be-45b5-88e7-ba8207aaeb46 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 14:45:35.735090 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:45:35.735090 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:45:35.735090 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:45:35.735090 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:45:35.735090 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:45:35.735090 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:45:35.735090 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:45:35.735090 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:45:35.735090 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:45:35.735090 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:45:35.735090 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:45:35.735090 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 14:45:35.735090 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 14:45:35.735090 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 14:45:35.735090 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 14:45:35.736213 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 14:45:35.736213 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 14:45:35.736213 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 14:45:35.736213 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 14:45:35.736213 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:45:35.736213 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:45:35.736213 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:45:35.736213 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:45:35.736213 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:45:35.736213 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:45:35.736213 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:45:35.736213 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application with Lock( Sep 01 14:45:35.736213 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:45:35.736213 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:45:35.736213 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:45:35.736213 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:45:35.737193 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:45:35.737193 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:45:35.737193 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:45:35.737193 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:45:35.737193 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 14:45:35.737193 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 14:45:35.737193 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 14:45:35.737193 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 14:45:35.737193 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 14:45:35.737193 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 14:45:35.737193 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 14:45:35.737193 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Sep 01 14:45:35.739357 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 147/301] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Sep 1 14:45:35 2022] GET /identity/v3/roles/member => generated 84 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:45:35.746448 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeab5786-8575-4e39-b595-f678747402ae None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:35.864500 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeab5786-8575-4e39-b595-f678747402ae None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:35.895260 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:35.907219 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4cab87-4f12-4a91-a499-35b1690184aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:35.907831 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4cab87-4f12-4a91-a499-35b1690184aa None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:35.908105 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4cab87-4f12-4a91-a499-35b1690184aa None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:35.908368 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4cab87-4f12-4a91-a499-35b1690184aa None admin] PATH_INFO: `/v3/roles` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:35.909098 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4cab87-4f12-4a91-a499-35b1690184aa None admin] RBAC: Adding query filter params (name=member) {{(pid=81342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:45:35.909394 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4cab87-4f12-4a91-a499-35b1690184aa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:35.910849 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4cab87-4f12-4a91-a499-35b1690184aa None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:35.913534 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a91b6eb-c6be-45b5-88e7-ba8207aaeb46 None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:35.921930 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 155/302] 10.222.0.33 () {56 vars in 1101 bytes} [Thu Sep 1 14:45:35 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:45:35.928289 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4cab87-4f12-4a91-a499-35b1690184aa None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:35.934189 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:36.032853 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4cab87-4f12-4a91-a499-35b1690184aa None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:36.042407 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a91b6eb-c6be-45b5-88e7-ba8207aaeb46 None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:36.066386 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11784ae2-281c-4cba-981c-36504819ffaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:36.067112 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11784ae2-281c-4cba-981c-36504819ffaa None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:36.067287 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11784ae2-281c-4cba-981c-36504819ffaa None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:36.067561 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11784ae2-281c-4cba-981c-36504819ffaa None admin] PATH_INFO: `/v3/users/0880b1f90dc44ff183565a21fec8f8a0` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:36.068711 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11784ae2-281c-4cba-981c-36504819ffaa None admin] RBAC: Authorizing `identity:get_user(user_id=0880b1f90dc44ff183565a21fec8f8a0)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:36.070155 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11784ae2-281c-4cba-981c-36504819ffaa None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:36.072016 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 156/303] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Sep 1 14:45:35 2022] GET /identity/v3/users/0880b1f90dc44ff183565a21fec8f8a0 => generated 293 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 14:45:36.073196 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57ee200-f75c-4645-b330-1d55b5f707b8 None None] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:36.073450 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57ee200-f75c-4645-b330-1d55b5f707b8 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:36.073720 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57ee200-f75c-4645-b330-1d55b5f707b8 None None] PATH_INFO: `/` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:36.075193 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 157/304] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 1 14:45:36 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:45:36.075910 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57ee200-f75c-4645-b330-1d55b5f707b8 None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:36.080702 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103693cd-4246-443f-a7a3-8fd9afda7186 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:36.081363 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103693cd-4246-443f-a7a3-8fd9afda7186 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:36.081635 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103693cd-4246-443f-a7a3-8fd9afda7186 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:36.081899 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103693cd-4246-443f-a7a3-8fd9afda7186 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:36.083053 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103693cd-4246-443f-a7a3-8fd9afda7186 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:36.085055 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103693cd-4246-443f-a7a3-8fd9afda7186 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:36.086635 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 148/305] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 1 14:45:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:45:36.087357 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103693cd-4246-443f-a7a3-8fd9afda7186 None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:36.137513 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:36.166783 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:36.181193 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57ee200-f75c-4645-b330-1d55b5f707b8 None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:36.184840 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103693cd-4246-443f-a7a3-8fd9afda7186 None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:36.214491 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740ea535-dad7-4b8f-a2c0-a76c8d2f76ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:36.215104 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740ea535-dad7-4b8f-a2c0-a76c8d2f76ad None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:36.215377 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740ea535-dad7-4b8f-a2c0-a76c8d2f76ad None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:36.215643 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740ea535-dad7-4b8f-a2c0-a76c8d2f76ad None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:36.218036 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99614bd3-6094-463d-a1f0-aae5e0dc2e00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:36.218705 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99614bd3-6094-463d-a1f0-aae5e0dc2e00 None admin] REQUEST_METHOD: `PUT` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:36.218985 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99614bd3-6094-463d-a1f0-aae5e0dc2e00 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:36.219258 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99614bd3-6094-463d-a1f0-aae5e0dc2e00 None admin] PATH_INFO: `/v3/system/users/0880b1f90dc44ff183565a21fec8f8a0/roles/5d1feef0e13d4874928bd86258c4e3b2` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:36.220672 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99614bd3-6094-463d-a1f0-aae5e0dc2e00 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=0880b1f90dc44ff183565a21fec8f8a0, role_id=5d1feef0e13d4874928bd86258c4e3b2)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:36.222066 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99614bd3-6094-463d-a1f0-aae5e0dc2e00 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:36.225903 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740ea535-dad7-4b8f-a2c0-a76c8d2f76ad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 14:45:36.230681 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application [None req-740ea535-dad7-4b8f-a2c0-a76c8d2f76ad None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 14:45:36.230681 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:45:36.230681 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:45:36.230681 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:45:36.230681 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:45:36.230681 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:45:36.230681 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:45:36.230681 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:45:36.230681 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:45:36.230681 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:45:36.230681 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:45:36.230681 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:45:36.230681 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 14:45:36.230681 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 14:45:36.230681 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 14:45:36.230681 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 14:45:36.246972 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 14:45:36.246972 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 14:45:36.246972 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 14:45:36.246972 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 14:45:36.246972 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:45:36.246972 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:45:36.246972 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:45:36.246972 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:45:36.246972 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:45:36.246972 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:45:36.246972 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:45:36.246972 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application with Lock( Sep 01 14:45:36.246972 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:45:36.246972 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:45:36.246972 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:45:36.246972 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:45:36.247988 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:45:36.247988 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:45:36.247988 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:45:36.247988 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:45:36.247988 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 14:45:36.247988 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 14:45:36.247988 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 14:45:36.247988 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 14:45:36.247988 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 14:45:36.247988 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 14:45:36.247988 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 14:45:36.247988 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Sep 01 14:45:36.248721 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 149/306] 10.222.0.33 () {58 vars in 1334 bytes} [Thu Sep 1 14:45:36 2022] PUT /identity/v3/system/users/0880b1f90dc44ff183565a21fec8f8a0/roles/5d1feef0e13d4874928bd86258c4e3b2 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:45:36.248721 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c980824-825c-4161-81b7-53638e16eb6f None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:36.248721 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c980824-825c-4161-81b7-53638e16eb6f None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:36.248721 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c980824-825c-4161-81b7-53638e16eb6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:36.249013 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 158/307] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Sep 1 14:45:36 2022] GET /identity/v3/roles/reader => generated 84 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:45:36.249013 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740ea535-dad7-4b8f-a2c0-a76c8d2f76ad None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:36.253371 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-4c980824-825c-4161-81b7-53638e16eb6f None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:45:36.254647 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-4c980824-825c-4161-81b7-53638e16eb6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:45:36.333274 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 150/308] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 1 14:45:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 14:45:36.334863 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c980824-825c-4161-81b7-53638e16eb6f None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:36.375739 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740ea535-dad7-4b8f-a2c0-a76c8d2f76ad None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:36.397352 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:36.420370 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2867777-c44b-4c2f-b87f-90c8055ecdda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:36.421261 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2867777-c44b-4c2f-b87f-90c8055ecdda None admin] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:36.421720 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2867777-c44b-4c2f-b87f-90c8055ecdda None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:36.422168 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2867777-c44b-4c2f-b87f-90c8055ecdda None admin] PATH_INFO: `/v3/groups` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:36.423013 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2867777-c44b-4c2f-b87f-90c8055ecdda None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:36.424863 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2867777-c44b-4c2f-b87f-90c8055ecdda None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:36.426110 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:36.451161 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 159/309] 10.222.0.33 () {60 vars in 1134 bytes} [Thu Sep 1 14:45:36 2022] POST /identity/v3/groups => generated 229 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:45:36.463181 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c980824-825c-4161-81b7-53638e16eb6f None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:36.506162 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26798eb1-7ba8-4980-a69e-5a30f75fba91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:36.506911 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26798eb1-7ba8-4980-a69e-5a30f75fba91 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:36.507194 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26798eb1-7ba8-4980-a69e-5a30f75fba91 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:36.507582 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26798eb1-7ba8-4980-a69e-5a30f75fba91 None admin] PATH_INFO: `/v3/roles` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:36.508332 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26798eb1-7ba8-4980-a69e-5a30f75fba91 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81341) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:45:36.508640 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26798eb1-7ba8-4980-a69e-5a30f75fba91 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:36.510076 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26798eb1-7ba8-4980-a69e-5a30f75fba91 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:36.520168 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 151/310] 10.222.0.33 () {56 vars in 1101 bytes} [Thu Sep 1 14:45:36 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:45:36.526462 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2867777-c44b-4c2f-b87f-90c8055ecdda None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:36.607834 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26798eb1-7ba8-4980-a69e-5a30f75fba91 None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:36.641858 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2867777-c44b-4c2f-b87f-90c8055ecdda None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:36.687998 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d93970-5613-4e29-875f-d3565549dd4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:36.689591 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d93970-5613-4e29-875f-d3565549dd4a None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:36.690072 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d93970-5613-4e29-875f-d3565549dd4a None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:36.690785 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d93970-5613-4e29-875f-d3565549dd4a None admin] PATH_INFO: `/v3/users/b4c009c99c384f8ea99b896ad585ce37` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:36.715311 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d93970-5613-4e29-875f-d3565549dd4a None admin] RBAC: Authorizing `identity:get_user(user_id=b4c009c99c384f8ea99b896ad585ce37)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:36.717202 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d93970-5613-4e29-875f-d3565549dd4a None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:36.719275 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 160/311] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Sep 1 14:45:36 2022] GET /identity/v3/users/b4c009c99c384f8ea99b896ad585ce37 => generated 293 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:45:36.725977 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d93970-5613-4e29-875f-d3565549dd4a None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:36.733737 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26798eb1-7ba8-4980-a69e-5a30f75fba91 None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:36.775122 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:36.785947 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c4a264-f336-4bba-90bd-15b75366ba2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:36.787008 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c4a264-f336-4bba-90bd-15b75366ba2f None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:36.787349 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c4a264-f336-4bba-90bd-15b75366ba2f None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:36.787545 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c4a264-f336-4bba-90bd-15b75366ba2f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:36.801198 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:36.809479 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c4a264-f336-4bba-90bd-15b75366ba2f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 14:45:36.816498 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application [None req-85c4a264-f336-4bba-90bd-15b75366ba2f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 14:45:36.816498 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:45:36.816498 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:45:36.816498 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:45:36.816498 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:45:36.816498 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:45:36.816498 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:45:36.816498 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:45:36.816498 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:45:36.816498 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:45:36.816498 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:45:36.816498 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:45:36.816498 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 14:45:36.816498 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 14:45:36.816498 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 14:45:36.816498 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 14:45:36.817400 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 14:45:36.817400 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 14:45:36.817400 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 14:45:36.817400 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 14:45:36.817400 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:45:36.817400 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:45:36.817400 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:45:36.817400 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:45:36.817400 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:45:36.817400 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:45:36.817400 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:45:36.817400 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application with Lock( Sep 01 14:45:36.817400 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:45:36.817400 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:45:36.817400 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:45:36.817400 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:45:36.818282 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:45:36.818282 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:45:36.818282 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:45:36.818282 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:45:36.818282 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 14:45:36.818282 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 14:45:36.818282 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 14:45:36.818282 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 14:45:36.818282 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 14:45:36.818282 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 14:45:36.818282 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 14:45:36.818282 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Sep 01 14:45:36.819959 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 152/312] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Sep 1 14:45:36 2022] GET /identity/v3/roles/reader => generated 84 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:45:36.826664 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c4a264-f336-4bba-90bd-15b75366ba2f None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:36.869844 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d93970-5613-4e29-875f-d3565549dd4a None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:36.911960 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f2e40a-359e-4442-aabc-3d9847246e65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:36.912567 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f2e40a-359e-4442-aabc-3d9847246e65 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:36.912839 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f2e40a-359e-4442-aabc-3d9847246e65 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:36.913102 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f2e40a-359e-4442-aabc-3d9847246e65 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:36.914479 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f2e40a-359e-4442-aabc-3d9847246e65 None admin] RBAC: Adding query filter params (role.id=869a5d710fb347a195e8f7b2cd3f0750, user.id=b4c009c99c384f8ea99b896ad585ce37, scope.system=all) {{(pid=81342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:45:36.914738 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f2e40a-359e-4442-aabc-3d9847246e65 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:36.916467 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f2e40a-359e-4442-aabc-3d9847246e65 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:36.922973 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 161/313] 10.222.0.33 () {56 vars in 1308 bytes} [Thu Sep 1 14:45:36 2022] GET /identity/v3/role_assignments?user.id=b4c009c99c384f8ea99b896ad585ce37&scope.system=all&role.id=869a5d710fb347a195e8f7b2cd3f0750 => generated 226 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 14:45:36.956542 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c4a264-f336-4bba-90bd-15b75366ba2f None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:36.995855 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2d3cf5-fb8f-4c07-ba90-56a874b41edf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:36.996830 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2d3cf5-fb8f-4c07-ba90-56a874b41edf None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:36.997301 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2d3cf5-fb8f-4c07-ba90-56a874b41edf None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:36.997764 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2d3cf5-fb8f-4c07-ba90-56a874b41edf None admin] PATH_INFO: `/v3/roles` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:36.998683 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2d3cf5-fb8f-4c07-ba90-56a874b41edf None admin] RBAC: Adding query filter params (name=reader) {{(pid=81341) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:45:36.999209 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2d3cf5-fb8f-4c07-ba90-56a874b41edf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:37.000809 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2d3cf5-fb8f-4c07-ba90-56a874b41edf None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:37.014778 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 153/314] 10.222.0.33 () {56 vars in 1101 bytes} [Thu Sep 1 14:45:36 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:45:37.018607 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f2e40a-359e-4442-aabc-3d9847246e65 None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:37.126482 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:37.129891 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f2e40a-359e-4442-aabc-3d9847246e65 None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:37.168423 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f36e6e-55f1-4f63-9257-d2c4f1444152 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:37.169299 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f36e6e-55f1-4f63-9257-d2c4f1444152 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:37.169713 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f36e6e-55f1-4f63-9257-d2c4f1444152 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:37.170119 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f36e6e-55f1-4f63-9257-d2c4f1444152 None admin] PATH_INFO: `/v3/users/67376baab7bd4cfa8a2e01f91535389e` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:37.171409 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f36e6e-55f1-4f63-9257-d2c4f1444152 None admin] RBAC: Authorizing `identity:get_user(user_id=67376baab7bd4cfa8a2e01f91535389e)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:37.173017 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f36e6e-55f1-4f63-9257-d2c4f1444152 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:37.175100 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 162/315] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Sep 1 14:45:37 2022] GET /identity/v3/users/67376baab7bd4cfa8a2e01f91535389e => generated 297 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:45:37.183662 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2d3cf5-fb8f-4c07-ba90-56a874b41edf None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:37.246490 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:37.291561 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2d3cf5-fb8f-4c07-ba90-56a874b41edf None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:37.326708 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a47165-c853-44de-99c9-a7a5c8e8bb01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:37.327313 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a47165-c853-44de-99c9-a7a5c8e8bb01 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:37.327592 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a47165-c853-44de-99c9-a7a5c8e8bb01 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:37.327867 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a47165-c853-44de-99c9-a7a5c8e8bb01 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:37.399206 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a47165-c853-44de-99c9-a7a5c8e8bb01 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:37.400685 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a47165-c853-44de-99c9-a7a5c8e8bb01 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:37.460222 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:37.502815 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 154/316] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Sep 1 14:45:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 14:45:37.511285 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f36e6e-55f1-4f63-9257-d2c4f1444152 None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:37.617631 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f36e6e-55f1-4f63-9257-d2c4f1444152 None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:37.656896 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d2c6f4-5915-40cf-b217-77dc865f3321 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:37.657528 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d2c6f4-5915-40cf-b217-77dc865f3321 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:37.657801 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d2c6f4-5915-40cf-b217-77dc865f3321 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:37.658082 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d2c6f4-5915-40cf-b217-77dc865f3321 None admin] PATH_INFO: `/v3/projects/b817b0f034a64d72914672463eed8f62` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:37.661115 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d2c6f4-5915-40cf-b217-77dc865f3321 None admin] RBAC: Authorizing `identity:get_project(project_id=b817b0f034a64d72914672463eed8f62)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:37.661115 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d2c6f4-5915-40cf-b217-77dc865f3321 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:37.663829 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 163/317] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Sep 1 14:45:37 2022] GET /identity/v3/projects/b817b0f034a64d72914672463eed8f62 => generated 302 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:45:37.669550 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a47165-c853-44de-99c9-a7a5c8e8bb01 None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:37.775037 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a47165-c853-44de-99c9-a7a5c8e8bb01 None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:37.809811 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e270ca5-2f9d-4237-b8f2-0b8d66c162ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:37.810446 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e270ca5-2f9d-4237-b8f2-0b8d66c162ac None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:37.810730 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e270ca5-2f9d-4237-b8f2-0b8d66c162ac None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:37.811007 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e270ca5-2f9d-4237-b8f2-0b8d66c162ac None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:37.811738 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e270ca5-2f9d-4237-b8f2-0b8d66c162ac None admin] RBAC: Adding query filter params (role.id=869a5d710fb347a195e8f7b2cd3f0750, scope.project.id=b817b0f034a64d72914672463eed8f62, user.id=67376baab7bd4cfa8a2e01f91535389e) {{(pid=81341) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:45:37.812047 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e270ca5-2f9d-4237-b8f2-0b8d66c162ac None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:37.813498 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e270ca5-2f9d-4237-b8f2-0b8d66c162ac None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:37.820257 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 155/318] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Sep 1 14:45:37 2022] GET /identity/v3/role_assignments?user.id=67376baab7bd4cfa8a2e01f91535389e&scope.project.id=b817b0f034a64d72914672463eed8f62&role.id=869a5d710fb347a195e8f7b2cd3f0750 => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:45:38.262261 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a54bb39-ec07-4a22-a321-76a6b0ba185a None None] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:38.262786 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a54bb39-ec07-4a22-a321-76a6b0ba185a None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:38.262786 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a54bb39-ec07-4a22-a321-76a6b0ba185a None None] PATH_INFO: `/` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:38.264185 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 164/319] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 1 14:45:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:45:38.269331 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4833cca1-9ca4-4dfc-af8b-220522c2e900 None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:38.269702 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4833cca1-9ca4-4dfc-af8b-220522c2e900 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:38.269942 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4833cca1-9ca4-4dfc-af8b-220522c2e900 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:38.288074 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-4833cca1-9ca4-4dfc-af8b-220522c2e900 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:45:38.289284 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-4833cca1-9ca4-4dfc-af8b-220522c2e900 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:45:38.308106 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:38.359603 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 156/320] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 1 14:45:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 14:45:38.457083 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a5ce54-ed7f-4ad9-a22c-524c2a9e2ab4 None None] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:38.457382 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a5ce54-ed7f-4ad9-a22c-524c2a9e2ab4 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:38.457641 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a5ce54-ed7f-4ad9-a22c-524c2a9e2ab4 None None] PATH_INFO: `/` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:38.459188 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 165/321] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 1 14:45:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:45:38.464099 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c5bfdb-87f6-43ed-93ef-8aeff9cfc9cd None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:38.464383 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c5bfdb-87f6-43ed-93ef-8aeff9cfc9cd None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:38.464677 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c5bfdb-87f6-43ed-93ef-8aeff9cfc9cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:38.483056 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-41c5bfdb-87f6-43ed-93ef-8aeff9cfc9cd None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:45:38.484227 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-41c5bfdb-87f6-43ed-93ef-8aeff9cfc9cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:45:38.550724 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 157/322] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 1 14:45:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 14:45:38.625510 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a5ce54-ed7f-4ad9-a22c-524c2a9e2ab4 None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:38.667574 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:38.735527 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a5ce54-ed7f-4ad9-a22c-524c2a9e2ab4 None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:38.769672 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7207409-bb55-4e96-a4cb-794428244ab7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:38.770332 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7207409-bb55-4e96-a4cb-794428244ab7 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:38.770651 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7207409-bb55-4e96-a4cb-794428244ab7 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:38.770916 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7207409-bb55-4e96-a4cb-794428244ab7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:38.780866 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7207409-bb55-4e96-a4cb-794428244ab7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 14:45:38.785660 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application [None req-c7207409-bb55-4e96-a4cb-794428244ab7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 14:45:38.785660 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:45:38.785660 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:45:38.785660 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:45:38.785660 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:45:38.785660 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:45:38.785660 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:45:38.785660 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:45:38.785660 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:45:38.785660 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:45:38.785660 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:45:38.785660 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:45:38.785660 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 14:45:38.785660 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 14:45:38.785660 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 14:45:38.785660 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 14:45:38.786563 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 14:45:38.786563 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 14:45:38.786563 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 14:45:38.786563 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 14:45:38.786563 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:45:38.786563 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:45:38.786563 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:45:38.786563 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:45:38.786563 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:45:38.786563 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:45:38.786563 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:45:38.786563 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application with Lock( Sep 01 14:45:38.786563 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:45:38.786563 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:45:38.786563 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:45:38.786563 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:45:38.791143 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:45:38.791143 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:45:38.791143 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:45:38.791143 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:45:38.791143 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 14:45:38.791143 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 14:45:38.791143 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 14:45:38.791143 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 14:45:38.791143 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 14:45:38.791143 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 14:45:38.791143 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 14:45:38.791143 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Sep 01 14:45:38.791143 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 166/323] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Sep 1 14:45:38 2022] GET /identity/v3/roles/member => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:45:38.793476 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c5bfdb-87f6-43ed-93ef-8aeff9cfc9cd None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:38.826187 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7207409-bb55-4e96-a4cb-794428244ab7 None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:38.881863 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:38.898221 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c5bfdb-87f6-43ed-93ef-8aeff9cfc9cd None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:38.924255 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:38.930934 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7207409-bb55-4e96-a4cb-794428244ab7 None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:38.931950 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5f3602-6c13-4952-b211-55cb0a563cd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:38.932553 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5f3602-6c13-4952-b211-55cb0a563cd4 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:38.932835 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5f3602-6c13-4952-b211-55cb0a563cd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:38.933107 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5f3602-6c13-4952-b211-55cb0a563cd4 None admin] PATH_INFO: `/v3/roles` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:38.933762 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5f3602-6c13-4952-b211-55cb0a563cd4 None admin] RBAC: Adding query filter params (name=member) {{(pid=81341) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:45:38.934061 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5f3602-6c13-4952-b211-55cb0a563cd4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:38.935487 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5f3602-6c13-4952-b211-55cb0a563cd4 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:38.943347 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 158/324] 10.222.0.33 () {56 vars in 1101 bytes} [Thu Sep 1 14:45:38 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:45:38.944627 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548e49df-5d5e-48b7-841a-9c8054ada0a5 None None] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:38.944908 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548e49df-5d5e-48b7-841a-9c8054ada0a5 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:38.945178 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548e49df-5d5e-48b7-841a-9c8054ada0a5 None None] PATH_INFO: `/` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:38.946773 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 159/325] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 1 14:45:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:45:38.948969 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548e49df-5d5e-48b7-841a-9c8054ada0a5 None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:38.969310 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2493100d-d5f5-490f-981c-c23caaed89dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:38.969931 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2493100d-d5f5-490f-981c-c23caaed89dc None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:38.970207 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2493100d-d5f5-490f-981c-c23caaed89dc None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:38.970507 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2493100d-d5f5-490f-981c-c23caaed89dc None admin] PATH_INFO: `/v3/domains/default` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:38.971603 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2493100d-d5f5-490f-981c-c23caaed89dc None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:38.973047 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2493100d-d5f5-490f-981c-c23caaed89dc None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:38.975201 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 167/326] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 1 14:45:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 14:45:38.975201 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daef464b-0430-4a84-b966-0546e6109291 None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:38.975201 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daef464b-0430-4a84-b966-0546e6109291 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:38.975201 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daef464b-0430-4a84-b966-0546e6109291 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:38.994389 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-daef464b-0430-4a84-b966-0546e6109291 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:45:38.995570 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-daef464b-0430-4a84-b966-0546e6109291 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:45:39.051925 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548e49df-5d5e-48b7-841a-9c8054ada0a5 None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:39.065362 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 168/327] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 1 14:45:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 14:45:39.066165 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daef464b-0430-4a84-b966-0546e6109291 None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:39.088208 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a5592a-5746-4ee0-b000-3cfa95839200 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:39.089378 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a5592a-5746-4ee0-b000-3cfa95839200 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:39.089855 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a5592a-5746-4ee0-b000-3cfa95839200 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:39.090555 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a5592a-5746-4ee0-b000-3cfa95839200 None admin] PATH_INFO: `/v3/users/0880b1f90dc44ff183565a21fec8f8a0` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:39.092556 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a5592a-5746-4ee0-b000-3cfa95839200 None admin] RBAC: Authorizing `identity:get_user(user_id=0880b1f90dc44ff183565a21fec8f8a0)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:39.094593 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a5592a-5746-4ee0-b000-3cfa95839200 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:39.101623 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 160/328] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Sep 1 14:45:38 2022] GET /identity/v3/users/0880b1f90dc44ff183565a21fec8f8a0 => generated 293 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 14:45:39.102655 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a5592a-5746-4ee0-b000-3cfa95839200 None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:39.118254 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:39.167294 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:39.176046 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daef464b-0430-4a84-b966-0546e6109291 None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:39.207785 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a5592a-5746-4ee0-b000-3cfa95839200 None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:39.211192 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a78a1b5-a67b-449c-8ecd-c312284f2c0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:39.211800 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a78a1b5-a67b-449c-8ecd-c312284f2c0f None admin] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:39.212080 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a78a1b5-a67b-449c-8ecd-c312284f2c0f None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:39.212350 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a78a1b5-a67b-449c-8ecd-c312284f2c0f None admin] PATH_INFO: `/v3/groups` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:39.212983 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a78a1b5-a67b-449c-8ecd-c312284f2c0f None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:39.214455 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a78a1b5-a67b-449c-8ecd-c312284f2c0f None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:39.230089 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 169/329] 10.222.0.33 () {60 vars in 1134 bytes} [Thu Sep 1 14:45:39 2022] POST /identity/v3/groups => generated 226 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:45:39.246092 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b38cfb-a130-4adf-bc82-ad7269dab750 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:39.246761 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b38cfb-a130-4adf-bc82-ad7269dab750 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:39.247090 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b38cfb-a130-4adf-bc82-ad7269dab750 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:39.247361 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b38cfb-a130-4adf-bc82-ad7269dab750 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:39.248113 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b38cfb-a130-4adf-bc82-ad7269dab750 None admin] RBAC: Adding query filter params (role.id=5d1feef0e13d4874928bd86258c4e3b2, user.id=0880b1f90dc44ff183565a21fec8f8a0, scope.system=all) {{(pid=81341) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:45:39.248476 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b38cfb-a130-4adf-bc82-ad7269dab750 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:39.250082 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b38cfb-a130-4adf-bc82-ad7269dab750 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:39.264108 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 161/330] 10.222.0.33 () {56 vars in 1308 bytes} [Thu Sep 1 14:45:39 2022] GET /identity/v3/role_assignments?user.id=0880b1f90dc44ff183565a21fec8f8a0&scope.system=all&role.id=5d1feef0e13d4874928bd86258c4e3b2 => generated 510 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 14:45:39.338737 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a78a1b5-a67b-449c-8ecd-c312284f2c0f None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:39.448887 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a78a1b5-a67b-449c-8ecd-c312284f2c0f None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:39.484197 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb21081-a967-4adb-a80f-72e216176aaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:39.484834 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb21081-a967-4adb-a80f-72e216176aaf None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:39.485109 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb21081-a967-4adb-a80f-72e216176aaf None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:39.485379 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb21081-a967-4adb-a80f-72e216176aaf None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:39.497125 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb21081-a967-4adb-a80f-72e216176aaf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 14:45:39.502516 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application [None req-feb21081-a967-4adb-a80f-72e216176aaf None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 14:45:39.502516 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:45:39.502516 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:45:39.502516 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:45:39.502516 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:45:39.502516 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:45:39.502516 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:45:39.502516 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:45:39.502516 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:45:39.502516 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:45:39.502516 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:45:39.502516 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:45:39.502516 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 14:45:39.502516 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 14:45:39.502516 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 14:45:39.502516 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 14:45:39.503424 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 14:45:39.503424 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 14:45:39.503424 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 14:45:39.503424 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 14:45:39.503424 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:45:39.503424 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:45:39.503424 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:45:39.503424 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:45:39.503424 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:45:39.503424 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:45:39.503424 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:45:39.503424 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application with Lock( Sep 01 14:45:39.503424 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:45:39.503424 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:45:39.503424 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:45:39.503424 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:45:39.504267 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:45:39.504267 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:45:39.504267 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:45:39.504267 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:45:39.504267 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 14:45:39.504267 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 14:45:39.504267 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 14:45:39.504267 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 14:45:39.504267 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 14:45:39.504267 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 14:45:39.504267 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 14:45:39.504267 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Sep 01 14:45:39.504267 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 170/331] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Sep 1 14:45:39 2022] GET /identity/v3/roles/reader => generated 84 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:45:39.510161 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b38cfb-a130-4adf-bc82-ad7269dab750 None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:39.583076 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:39.616404 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b38cfb-a130-4adf-bc82-ad7269dab750 None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:39.658935 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fc3be3-3585-4b9b-bd76-9702a3611f1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:39.660018 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fc3be3-3585-4b9b-bd76-9702a3611f1e None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:39.660515 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fc3be3-3585-4b9b-bd76-9702a3611f1e None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:39.660981 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fc3be3-3585-4b9b-bd76-9702a3611f1e None admin] PATH_INFO: `/v3/roles` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:39.661916 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fc3be3-3585-4b9b-bd76-9702a3611f1e None admin] RBAC: Adding query filter params (name=reader) {{(pid=81341) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:45:39.662435 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fc3be3-3585-4b9b-bd76-9702a3611f1e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:39.664094 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fc3be3-3585-4b9b-bd76-9702a3611f1e None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:39.673630 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 162/332] 10.222.0.33 () {56 vars in 1101 bytes} [Thu Sep 1 14:45:39 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:45:39.679141 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb21081-a967-4adb-a80f-72e216176aaf None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:39.698891 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:39.795878 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb21081-a967-4adb-a80f-72e216176aaf None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:39.834593 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf956396-83c3-4a7d-9d3b-db96ccab7670 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:39.835290 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf956396-83c3-4a7d-9d3b-db96ccab7670 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:39.835568 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf956396-83c3-4a7d-9d3b-db96ccab7670 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:39.835835 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf956396-83c3-4a7d-9d3b-db96ccab7670 None admin] PATH_INFO: `/v3/users/b4c009c99c384f8ea99b896ad585ce37` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:39.837033 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf956396-83c3-4a7d-9d3b-db96ccab7670 None admin] RBAC: Authorizing `identity:get_user(user_id=b4c009c99c384f8ea99b896ad585ce37)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:39.838522 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf956396-83c3-4a7d-9d3b-db96ccab7670 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:39.840489 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 171/333] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Sep 1 14:45:39 2022] GET /identity/v3/users/b4c009c99c384f8ea99b896ad585ce37 => generated 293 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:45:39.845673 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fc3be3-3585-4b9b-bd76-9702a3611f1e None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:39.952060 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fc3be3-3585-4b9b-bd76-9702a3611f1e None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:39.986568 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7061e14-db22-4d42-9e6f-29c16fb3fec7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:39.987278 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7061e14-db22-4d42-9e6f-29c16fb3fec7 None admin] REQUEST_METHOD: `PUT` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:39.987562 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7061e14-db22-4d42-9e6f-29c16fb3fec7 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:39.987824 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7061e14-db22-4d42-9e6f-29c16fb3fec7 None admin] PATH_INFO: `/v3/system/users/b4c009c99c384f8ea99b896ad585ce37/roles/869a5d710fb347a195e8f7b2cd3f0750` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:39.989272 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7061e14-db22-4d42-9e6f-29c16fb3fec7 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=b4c009c99c384f8ea99b896ad585ce37, role_id=869a5d710fb347a195e8f7b2cd3f0750)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:39.991511 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7061e14-db22-4d42-9e6f-29c16fb3fec7 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:39.993102 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:40.001877 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 163/334] 10.222.0.33 () {58 vars in 1334 bytes} [Thu Sep 1 14:45:39 2022] PUT /identity/v3/system/users/b4c009c99c384f8ea99b896ad585ce37/roles/869a5d710fb347a195e8f7b2cd3f0750 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 14:45:41.644483 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5c2392-5111-4278-a6c9-bc1311e86541 None None] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:41.644988 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5c2392-5111-4278-a6c9-bc1311e86541 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:41.644988 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5c2392-5111-4278-a6c9-bc1311e86541 None None] PATH_INFO: `/` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:41.646669 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 172/335] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 1 14:45:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:45:41.658051 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f358dc-2121-4870-a814-4a5aa2530e44 None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:41.658640 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f358dc-2121-4870-a814-4a5aa2530e44 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:41.658640 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f358dc-2121-4870-a814-4a5aa2530e44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:41.682247 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-a3f358dc-2121-4870-a814-4a5aa2530e44 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:45:41.683874 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-a3f358dc-2121-4870-a814-4a5aa2530e44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:45:41.770160 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 164/336] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 1 14:45:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 14:45:42.325880 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5c2392-5111-4278-a6c9-bc1311e86541 None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:42.343973 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44306401-fe2c-41fc-9c3a-b175e2836060 None None] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:42.344474 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44306401-fe2c-41fc-9c3a-b175e2836060 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:42.344889 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44306401-fe2c-41fc-9c3a-b175e2836060 None None] PATH_INFO: `/` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:42.346466 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 165/337] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 1 14:45:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:45:42.358260 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fc6c4b-ecdf-45fe-9dbc-92aefbe3fc11 None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:42.358260 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fc6c4b-ecdf-45fe-9dbc-92aefbe3fc11 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:42.358260 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fc6c4b-ecdf-45fe-9dbc-92aefbe3fc11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:42.388580 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-46fc6c4b-ecdf-45fe-9dbc-92aefbe3fc11 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:45:42.390145 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-46fc6c4b-ecdf-45fe-9dbc-92aefbe3fc11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:45:42.454703 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:42.487342 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5c2392-5111-4278-a6c9-bc1311e86541 None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:42.538471 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3d27b4-6b74-458a-9f9c-00977c561cce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:42.539108 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3d27b4-6b74-458a-9f9c-00977c561cce None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:42.539387 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3d27b4-6b74-458a-9f9c-00977c561cce None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:42.539655 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3d27b4-6b74-458a-9f9c-00977c561cce None admin] PATH_INFO: `/v3/roles/member` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:42.554368 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 166/338] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 1 14:45:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 14:45:42.555779 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafaaac1-18cb-441c-9dbc-3f3f58eca6c6 None None] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:42.556065 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafaaac1-18cb-441c-9dbc-3f3f58eca6c6 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:42.556341 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafaaac1-18cb-441c-9dbc-3f3f58eca6c6 None None] PATH_INFO: `/` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:42.561135 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 167/339] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 1 14:45:42 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:45:42.565657 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b398249-a331-40ab-bfb3-af3ecec2cead None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:42.565937 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b398249-a331-40ab-bfb3-af3ecec2cead None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:42.566203 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b398249-a331-40ab-bfb3-af3ecec2cead None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:42.570333 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3d27b4-6b74-458a-9f9c-00977c561cce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 14:45:42.575437 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application [None req-eb3d27b4-6b74-458a-9f9c-00977c561cce None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 14:45:42.575437 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:45:42.575437 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:45:42.575437 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:45:42.575437 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:45:42.575437 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:45:42.575437 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:45:42.575437 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:45:42.575437 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:45:42.575437 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:45:42.575437 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:45:42.575437 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:45:42.575437 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 14:45:42.575437 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 14:45:42.575437 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 14:45:42.575437 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 14:45:42.576402 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 14:45:42.576402 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 14:45:42.576402 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 14:45:42.576402 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 14:45:42.576402 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:45:42.576402 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:45:42.576402 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:45:42.576402 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:45:42.576402 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:45:42.576402 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:45:42.576402 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:45:42.576402 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application with Lock( Sep 01 14:45:42.576402 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:45:42.576402 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:45:42.576402 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:45:42.576402 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:45:42.577246 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:45:42.577246 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:45:42.577246 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:45:42.577246 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:45:42.577246 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 14:45:42.577246 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 14:45:42.577246 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 14:45:42.577246 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 14:45:42.577246 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 14:45:42.577246 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 14:45:42.577246 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 14:45:42.577246 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Sep 01 14:45:42.590423 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 173/340] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Sep 1 14:45:42 2022] GET /identity/v3/roles/member => generated 84 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:45:42.595231 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3d27b4-6b74-458a-9f9c-00977c561cce None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:42.598857 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-1b398249-a331-40ab-bfb3-af3ecec2cead None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:45:42.600117 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-1b398249-a331-40ab-bfb3-af3ecec2cead None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:45:42.678122 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:42.701293 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 168/341] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 1 14:45:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 14:45:42.807548 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3d27b4-6b74-458a-9f9c-00977c561cce None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:42.852654 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a20e66-d3f1-45c3-aae4-cd92f64248ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:42.853616 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a20e66-d3f1-45c3-aae4-cd92f64248ff None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:42.854087 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a20e66-d3f1-45c3-aae4-cd92f64248ff None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:42.854568 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a20e66-d3f1-45c3-aae4-cd92f64248ff None admin] PATH_INFO: `/v3/roles` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:42.855429 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a20e66-d3f1-45c3-aae4-cd92f64248ff None admin] RBAC: Adding query filter params (name=member) {{(pid=81342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:45:42.855937 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a20e66-d3f1-45c3-aae4-cd92f64248ff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:42.857517 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a20e66-d3f1-45c3-aae4-cd92f64248ff None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:42.870318 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 174/342] 10.222.0.33 () {56 vars in 1101 bytes} [Thu Sep 1 14:45:42 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:45:42.876197 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b398249-a331-40ab-bfb3-af3ecec2cead None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:42.967656 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a20e66-d3f1-45c3-aae4-cd92f64248ff None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:42.974297 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:43.029862 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b398249-a331-40ab-bfb3-af3ecec2cead None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:43.096349 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b00a4e-eff8-4c95-96b3-df7c288e365a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:43.096971 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b00a4e-eff8-4c95-96b3-df7c288e365a None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:43.097260 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b00a4e-eff8-4c95-96b3-df7c288e365a None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:43.097538 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b00a4e-eff8-4c95-96b3-df7c288e365a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:43.127612 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a20e66-d3f1-45c3-aae4-cd92f64248ff None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:43.168472 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bef0b38-8b26-415d-b369-a78b6d7b978d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:43.169444 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bef0b38-8b26-415d-b369-a78b6d7b978d None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:43.169915 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bef0b38-8b26-415d-b369-a78b6d7b978d None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:43.170471 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bef0b38-8b26-415d-b369-a78b6d7b978d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:43.183934 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bef0b38-8b26-415d-b369-a78b6d7b978d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 14:45:43.191374 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application [None req-4bef0b38-8b26-415d-b369-a78b6d7b978d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 14:45:43.191374 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:45:43.191374 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:45:43.191374 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:45:43.191374 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:45:43.191374 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:45:43.191374 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:45:43.191374 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:45:43.191374 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:45:43.191374 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:45:43.191374 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:45:43.191374 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:45:43.191374 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 14:45:43.191374 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 14:45:43.191374 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 14:45:43.191374 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 14:45:43.192350 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 14:45:43.192350 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 14:45:43.192350 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 14:45:43.192350 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 14:45:43.192350 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:45:43.192350 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:45:43.192350 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:45:43.192350 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:45:43.192350 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:45:43.192350 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:45:43.192350 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:45:43.192350 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application with Lock( Sep 01 14:45:43.192350 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:45:43.192350 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:45:43.192350 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:45:43.192350 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:45:43.193216 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:45:43.193216 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:45:43.193216 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:45:43.193216 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:45:43.193216 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 14:45:43.193216 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 14:45:43.193216 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 14:45:43.193216 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 14:45:43.193216 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 14:45:43.193216 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 14:45:43.193216 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 14:45:43.193216 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Sep 01 14:45:43.198687 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 175/343] 10.222.0.33 () {56 vars in 1114 bytes} [Thu Sep 1 14:45:42 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:45:43.198687 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b531b6f-3dac-4aa9-b517-4d7e219c4be3 None None] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:43.198687 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b531b6f-3dac-4aa9-b517-4d7e219c4be3 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:43.198687 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b531b6f-3dac-4aa9-b517-4d7e219c4be3 None None] PATH_INFO: `/` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:43.198687 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 176/344] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 1 14:45:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:45:43.207522 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:43.208404 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b531b6f-3dac-4aa9-b517-4d7e219c4be3 None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:43.226054 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b00a4e-eff8-4c95-96b3-df7c288e365a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:43.227692 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b00a4e-eff8-4c95-96b3-df7c288e365a None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:43.299268 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:43.352206 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 169/345] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Sep 1 14:45:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 476 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 14:45:43.353241 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b00a4e-eff8-4c95-96b3-df7c288e365a None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:43.355697 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b531b6f-3dac-4aa9-b517-4d7e219c4be3 None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:43.402958 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90616c5-905f-4970-b866-970a79e1f24a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:43.403485 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90616c5-905f-4970-b866-970a79e1f24a None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:43.403767 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90616c5-905f-4970-b866-970a79e1f24a None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:43.404029 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90616c5-905f-4970-b866-970a79e1f24a None admin] PATH_INFO: `/v3/roles` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:43.404691 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90616c5-905f-4970-b866-970a79e1f24a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:45:43.404995 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90616c5-905f-4970-b866-970a79e1f24a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:43.406537 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90616c5-905f-4970-b866-970a79e1f24a None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:43.415360 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 177/346] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Sep 1 14:45:43 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:45:43.417144 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41c6227-28b0-402f-ab2a-a07865fe3ef2 None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:43.417414 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41c6227-28b0-402f-ab2a-a07865fe3ef2 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:43.417683 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41c6227-28b0-402f-ab2a-a07865fe3ef2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:43.439414 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-a41c6227-28b0-402f-ab2a-a07865fe3ef2 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:45:43.440984 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-a41c6227-28b0-402f-ab2a-a07865fe3ef2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:45:43.455315 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:43.472225 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b00a4e-eff8-4c95-96b3-df7c288e365a None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:43.515407 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3901a075-2bda-4520-a6e6-77026f6f10d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:43.516052 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3901a075-2bda-4520-a6e6-77026f6f10d1 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:43.516331 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3901a075-2bda-4520-a6e6-77026f6f10d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:43.516604 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3901a075-2bda-4520-a6e6-77026f6f10d1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:43.522924 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 178/347] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 1 14:45:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 14:45:43.523578 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41c6227-28b0-402f-ab2a-a07865fe3ef2 None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:43.540143 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3901a075-2bda-4520-a6e6-77026f6f10d1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 14:45:43.545118 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application [None req-3901a075-2bda-4520-a6e6-77026f6f10d1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 14:45:43.545118 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:45:43.545118 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:45:43.545118 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:45:43.545118 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:45:43.545118 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:45:43.545118 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:45:43.545118 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:45:43.545118 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:45:43.545118 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:45:43.545118 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:45:43.545118 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:45:43.545118 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 14:45:43.545118 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 14:45:43.545118 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 14:45:43.545118 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 14:45:43.546128 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 14:45:43.546128 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 14:45:43.546128 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 14:45:43.546128 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 14:45:43.546128 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:45:43.546128 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:45:43.546128 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:45:43.546128 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:45:43.546128 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:45:43.546128 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:45:43.546128 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:45:43.546128 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application with Lock( Sep 01 14:45:43.546128 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:45:43.546128 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:45:43.546128 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:45:43.546128 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:45:43.547065 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:45:43.547065 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:45:43.547065 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:45:43.547065 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:45:43.547065 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 14:45:43.547065 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 14:45:43.547065 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 14:45:43.547065 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 14:45:43.547065 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 14:45:43.547065 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 14:45:43.547065 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 14:45:43.547065 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Sep 01 14:45:43.547065 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 170/348] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Sep 1 14:45:43 2022] GET /identity/v3/roles/member => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:45:43.553456 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be04da02-bc32-4198-b88b-59f3f03568f3 None None] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:43.553456 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be04da02-bc32-4198-b88b-59f3f03568f3 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:43.553456 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be04da02-bc32-4198-b88b-59f3f03568f3 None None] PATH_INFO: `/` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:43.553456 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 171/349] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 1 14:45:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:45:43.553456 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9bb764-059b-473c-9781-f1837b9a0c6b None None] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:43.553833 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9bb764-059b-473c-9781-f1837b9a0c6b None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:43.553902 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9bb764-059b-473c-9781-f1837b9a0c6b None None] PATH_INFO: `/` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:43.556481 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 172/350] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 1 14:45:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:45:43.557137 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9bb764-059b-473c-9781-f1837b9a0c6b None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:43.596447 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:43.657226 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41c6227-28b0-402f-ab2a-a07865fe3ef2 None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:43.685664 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9bb764-059b-473c-9781-f1837b9a0c6b None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:43.707611 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ccf642-a022-4806-814a-babdc759ef0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:43.708529 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ccf642-a022-4806-814a-babdc759ef0d None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:43.709523 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ccf642-a022-4806-814a-babdc759ef0d None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:43.709969 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ccf642-a022-4806-814a-babdc759ef0d None admin] PATH_INFO: `/v3/projects/1f696f55becc4e879b6d1bac9d93d055` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:43.711538 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ccf642-a022-4806-814a-babdc759ef0d None admin] RBAC: Authorizing `identity:get_project(project_id=1f696f55becc4e879b6d1bac9d93d055)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:43.713025 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ccf642-a022-4806-814a-babdc759ef0d None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:43.714829 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 179/351] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Sep 1 14:45:43 2022] GET /identity/v3/projects/1f696f55becc4e879b6d1bac9d93d055 => generated 298 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:45:43.715576 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ccf642-a022-4806-814a-babdc759ef0d None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:43.734160 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69fa64a-d595-4f8c-a963-2bd202d33e30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:43.734790 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69fa64a-d595-4f8c-a963-2bd202d33e30 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:43.735131 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69fa64a-d595-4f8c-a963-2bd202d33e30 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:43.735396 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69fa64a-d595-4f8c-a963-2bd202d33e30 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:43.760510 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:43.823092 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69fa64a-d595-4f8c-a963-2bd202d33e30 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:43.824624 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69fa64a-d595-4f8c-a963-2bd202d33e30 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:43.854772 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ccf642-a022-4806-814a-babdc759ef0d None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:43.881208 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:43.894718 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f57d0c-b3f2-48a1-9993-523f4be0a486 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:43.895225 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f57d0c-b3f2-48a1-9993-523f4be0a486 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:43.895508 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f57d0c-b3f2-48a1-9993-523f4be0a486 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:43.895780 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f57d0c-b3f2-48a1-9993-523f4be0a486 None admin] PATH_INFO: `/v3/roles` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:43.897034 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f57d0c-b3f2-48a1-9993-523f4be0a486 None admin] RBAC: Adding query filter params (name=member) {{(pid=81342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:45:43.898066 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f57d0c-b3f2-48a1-9993-523f4be0a486 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:43.898737 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f57d0c-b3f2-48a1-9993-523f4be0a486 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:43.907326 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 180/352] 10.222.0.33 () {56 vars in 1101 bytes} [Thu Sep 1 14:45:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:45:43.910345 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa62c59-a7d2-4d3a-8dc2-a46ee7661540 None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:43.910345 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa62c59-a7d2-4d3a-8dc2-a46ee7661540 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:43.910345 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa62c59-a7d2-4d3a-8dc2-a46ee7661540 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:43.927973 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-bfa62c59-a7d2-4d3a-8dc2-a46ee7661540 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:45:43.929152 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-bfa62c59-a7d2-4d3a-8dc2-a46ee7661540 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:45:43.949323 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 173/353] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Sep 1 14:45:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 14:45:43.950612 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9847e920-b1d4-419b-a781-91a6cca4d565 None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:43.950954 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9847e920-b1d4-419b-a781-91a6cca4d565 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:43.951220 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9847e920-b1d4-419b-a781-91a6cca4d565 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:43.970882 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-9847e920-b1d4-419b-a781-91a6cca4d565 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:45:43.972091 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-9847e920-b1d4-419b-a781-91a6cca4d565 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:45:44.010330 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 181/354] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 1 14:45:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 14:45:44.010330 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa62c59-a7d2-4d3a-8dc2-a46ee7661540 None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:44.027069 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:44.065964 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 174/355] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 1 14:45:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 14:45:44.066936 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9847e920-b1d4-419b-a781-91a6cca4d565 None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:44.126161 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa62c59-a7d2-4d3a-8dc2-a46ee7661540 None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:44.147658 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:44.165633 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420b6e99-f151-49e7-b38b-5c7ffb37b50f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:44.166311 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420b6e99-f151-49e7-b38b-5c7ffb37b50f None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:44.166663 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420b6e99-f151-49e7-b38b-5c7ffb37b50f None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:44.167000 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420b6e99-f151-49e7-b38b-5c7ffb37b50f None admin] PATH_INFO: `/v3/groups/2bca6ad19b2849619ed210863be8b696` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:44.173915 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420b6e99-f151-49e7-b38b-5c7ffb37b50f None admin] RBAC: Authorizing `identity:get_group(group_id=2bca6ad19b2849619ed210863be8b696)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:44.175647 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420b6e99-f151-49e7-b38b-5c7ffb37b50f None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:44.177532 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 182/356] 10.222.0.33 () {56 vars in 1180 bytes} [Thu Sep 1 14:45:44 2022] GET /identity/v3/groups/2bca6ad19b2849619ed210863be8b696 => generated 226 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:45:44.178563 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420b6e99-f151-49e7-b38b-5c7ffb37b50f None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:44.181792 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9847e920-b1d4-419b-a781-91a6cca4d565 None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:44.220874 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b520121-9dbf-41b5-a3d9-0ac16bf4920d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:44.221505 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b520121-9dbf-41b5-a3d9-0ac16bf4920d None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:44.221777 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b520121-9dbf-41b5-a3d9-0ac16bf4920d None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:44.222040 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b520121-9dbf-41b5-a3d9-0ac16bf4920d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:44.232793 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b520121-9dbf-41b5-a3d9-0ac16bf4920d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 14:45:44.238268 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application [None req-1b520121-9dbf-41b5-a3d9-0ac16bf4920d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 14:45:44.238268 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:45:44.238268 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:45:44.238268 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:45:44.238268 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:45:44.238268 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:45:44.238268 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:45:44.238268 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:45:44.238268 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:45:44.238268 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:45:44.238268 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:45:44.238268 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:45:44.238268 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 14:45:44.238268 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 14:45:44.238268 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 14:45:44.238268 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 14:45:44.239358 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 14:45:44.239358 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 14:45:44.239358 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 14:45:44.239358 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 14:45:44.239358 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:45:44.239358 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:45:44.239358 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:45:44.239358 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:45:44.239358 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:45:44.239358 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:45:44.239358 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:45:44.239358 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application with Lock( Sep 01 14:45:44.239358 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:45:44.239358 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:45:44.239358 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:45:44.239358 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:45:44.240827 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:45:44.240827 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:45:44.240827 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:45:44.240827 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:45:44.240827 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 14:45:44.240827 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 14:45:44.240827 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 14:45:44.240827 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 14:45:44.240827 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 14:45:44.240827 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 14:45:44.240827 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 14:45:44.240827 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Sep 01 14:45:44.240827 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 175/357] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Sep 1 14:45:44 2022] GET /identity/v3/roles/reader => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:45:44.240827 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b520121-9dbf-41b5-a3d9-0ac16bf4920d None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:44.283186 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:44.294895 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420b6e99-f151-49e7-b38b-5c7ffb37b50f None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:44.333753 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cc0c49-5998-4606-8900-88c7fda9589e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:44.334388 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cc0c49-5998-4606-8900-88c7fda9589e None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:44.334665 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cc0c49-5998-4606-8900-88c7fda9589e None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:44.334933 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cc0c49-5998-4606-8900-88c7fda9589e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:44.351033 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b520121-9dbf-41b5-a3d9-0ac16bf4920d None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:44.392244 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:44.393869 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadd41ab-6630-4ec3-81d9-4db4600c32dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:44.394655 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadd41ab-6630-4ec3-81d9-4db4600c32dc None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:44.394957 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadd41ab-6630-4ec3-81d9-4db4600c32dc None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:44.395233 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadd41ab-6630-4ec3-81d9-4db4600c32dc None admin] PATH_INFO: `/v3/projects/1f696f55becc4e879b6d1bac9d93d055` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:44.397085 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadd41ab-6630-4ec3-81d9-4db4600c32dc None admin] RBAC: Authorizing `identity:get_project(project_id=1f696f55becc4e879b6d1bac9d93d055)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:44.398599 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadd41ab-6630-4ec3-81d9-4db4600c32dc None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:44.400242 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 176/358] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Sep 1 14:45:44 2022] GET /identity/v3/projects/1f696f55becc4e879b6d1bac9d93d055 => generated 298 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:45:44.402987 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadd41ab-6630-4ec3-81d9-4db4600c32dc None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:44.413280 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5cc0c49-5998-4606-8900-88c7fda9589e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:44.414789 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5cc0c49-5998-4606-8900-88c7fda9589e None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:44.522794 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadd41ab-6630-4ec3-81d9-4db4600c32dc None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:44.530397 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:44.532975 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 183/359] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Sep 1 14:45:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 14:45:44.533653 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cc0c49-5998-4606-8900-88c7fda9589e None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:44.564466 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5d0eb3-0f16-4f50-baec-338a6571b1c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:44.565548 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5d0eb3-0f16-4f50-baec-338a6571b1c1 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:44.565548 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5d0eb3-0f16-4f50-baec-338a6571b1c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:44.565548 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5d0eb3-0f16-4f50-baec-338a6571b1c1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:44.566340 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5d0eb3-0f16-4f50-baec-338a6571b1c1 None admin] RBAC: Adding query filter params (group.id=2bca6ad19b2849619ed210863be8b696, role.id=5d1feef0e13d4874928bd86258c4e3b2, scope.project.id=1f696f55becc4e879b6d1bac9d93d055) {{(pid=81341) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:45:44.566646 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5d0eb3-0f16-4f50-baec-338a6571b1c1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:44.568614 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5d0eb3-0f16-4f50-baec-338a6571b1c1 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:44.576285 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 177/360] 10.222.0.33 () {56 vars in 1376 bytes} [Thu Sep 1 14:45:44 2022] GET /identity/v3/role_assignments?group.id=2bca6ad19b2849619ed210863be8b696&scope.project.id=1f696f55becc4e879b6d1bac9d93d055&role.id=5d1feef0e13d4874928bd86258c4e3b2 => generated 260 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:45:44.577082 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5d0eb3-0f16-4f50-baec-338a6571b1c1 None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:44.631029 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:44.659536 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cc0c49-5998-4606-8900-88c7fda9589e None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:44.711703 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4ed02a-2261-45a9-93dd-8f68cf0aa18f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:44.712333 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4ed02a-2261-45a9-93dd-8f68cf0aa18f None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:44.712612 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4ed02a-2261-45a9-93dd-8f68cf0aa18f None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:44.712929 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4ed02a-2261-45a9-93dd-8f68cf0aa18f None admin] PATH_INFO: `/v3/roles` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:44.713759 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4ed02a-2261-45a9-93dd-8f68cf0aa18f None admin] RBAC: Adding query filter params (name=reader) {{(pid=81342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:45:44.714374 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4ed02a-2261-45a9-93dd-8f68cf0aa18f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:44.715943 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4ed02a-2261-45a9-93dd-8f68cf0aa18f None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:44.730352 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 184/361] 10.222.0.33 () {56 vars in 1101 bytes} [Thu Sep 1 14:45:44 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:45:44.730537 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5d0eb3-0f16-4f50-baec-338a6571b1c1 None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:44.730933 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4ed02a-2261-45a9-93dd-8f68cf0aa18f None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:44.776159 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f880e785-59c7-4cc3-a9f2-6b2e1bf220d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:44.777572 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f880e785-59c7-4cc3-a9f2-6b2e1bf220d0 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:44.778028 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f880e785-59c7-4cc3-a9f2-6b2e1bf220d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:44.779445 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f880e785-59c7-4cc3-a9f2-6b2e1bf220d0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:44.790889 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f880e785-59c7-4cc3-a9f2-6b2e1bf220d0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 14:45:44.797817 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application [None req-f880e785-59c7-4cc3-a9f2-6b2e1bf220d0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 14:45:44.797817 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:45:44.797817 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:45:44.797817 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:45:44.797817 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:45:44.797817 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:45:44.797817 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:45:44.797817 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:45:44.797817 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:45:44.797817 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:45:44.797817 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:45:44.797817 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:45:44.797817 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 14:45:44.797817 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 14:45:44.797817 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 14:45:44.797817 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 14:45:44.798819 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 14:45:44.798819 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 14:45:44.798819 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 14:45:44.798819 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 14:45:44.798819 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:45:44.798819 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:45:44.798819 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:45:44.798819 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:45:44.798819 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:45:44.798819 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:45:44.798819 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:45:44.798819 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application with Lock( Sep 01 14:45:44.798819 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:45:44.798819 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:45:44.798819 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:45:44.798819 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:45:44.799642 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:45:44.799642 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:45:44.799642 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:45:44.799642 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:45:44.799642 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 14:45:44.799642 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 14:45:44.799642 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 14:45:44.799642 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 14:45:44.799642 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 14:45:44.799642 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 14:45:44.799642 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 14:45:44.799642 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Sep 01 14:45:44.802931 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 178/362] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Sep 1 14:45:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:45:44.802931 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f880e785-59c7-4cc3-a9f2-6b2e1bf220d0 None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:44.848941 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:44.863967 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4ed02a-2261-45a9-93dd-8f68cf0aa18f None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:44.915451 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786bb450-cc92-425c-91af-46fee0f175e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:44.916084 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786bb450-cc92-425c-91af-46fee0f175e0 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:44.916359 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786bb450-cc92-425c-91af-46fee0f175e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:44.916625 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786bb450-cc92-425c-91af-46fee0f175e0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:44.930384 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786bb450-cc92-425c-91af-46fee0f175e0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 14:45:44.937835 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application [None req-786bb450-cc92-425c-91af-46fee0f175e0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 14:45:44.937835 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:45:44.937835 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:45:44.937835 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:45:44.937835 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:45:44.937835 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:45:44.937835 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:45:44.937835 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:45:44.937835 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:45:44.937835 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:45:44.937835 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:45:44.937835 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:45:44.937835 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 14:45:44.937835 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 14:45:44.937835 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 14:45:44.937835 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 14:45:44.938794 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 14:45:44.938794 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 14:45:44.938794 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 14:45:44.938794 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 14:45:44.938794 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:45:44.938794 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:45:44.938794 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:45:44.938794 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:45:44.938794 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:45:44.938794 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:45:44.938794 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:45:44.938794 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application with Lock( Sep 01 14:45:44.938794 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:45:44.938794 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:45:44.938794 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:45:44.938794 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:45:44.939624 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:45:44.939624 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:45:44.939624 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:45:44.939624 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:45:44.939624 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 14:45:44.939624 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 14:45:44.939624 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 14:45:44.939624 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 14:45:44.939624 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 14:45:44.939624 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 14:45:44.939624 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 14:45:44.939624 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Sep 01 14:45:44.939624 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 185/363] 10.222.0.33 () {56 vars in 1114 bytes} [Thu Sep 1 14:45:44 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:45:44.940778 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786bb450-cc92-425c-91af-46fee0f175e0 None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:44.944869 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:44.949449 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f880e785-59c7-4cc3-a9f2-6b2e1bf220d0 None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:45.001473 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cdb4ed-aa50-40b3-b134-76edba8fa5df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:45.001895 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cdb4ed-aa50-40b3-b134-76edba8fa5df None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:45.002103 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cdb4ed-aa50-40b3-b134-76edba8fa5df None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:45.004096 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cdb4ed-aa50-40b3-b134-76edba8fa5df None admin] PATH_INFO: `/v3/groups/2bca6ad19b2849619ed210863be8b696` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:45.005398 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7cdb4ed-aa50-40b3-b134-76edba8fa5df None admin] RBAC: Authorizing `identity:get_group(group_id=2bca6ad19b2849619ed210863be8b696)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:45.008494 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7cdb4ed-aa50-40b3-b134-76edba8fa5df None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:45.012733 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 179/364] 10.222.0.33 () {56 vars in 1180 bytes} [Thu Sep 1 14:45:44 2022] GET /identity/v3/groups/2bca6ad19b2849619ed210863be8b696 => generated 226 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:45:45.014907 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cdb4ed-aa50-40b3-b134-76edba8fa5df None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:45.084733 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786bb450-cc92-425c-91af-46fee0f175e0 None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:45.125212 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:45.131651 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5326a8d3-1bb0-4b78-9b0c-a8439dbdd095 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:45.132630 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5326a8d3-1bb0-4b78-9b0c-a8439dbdd095 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:45.133044 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5326a8d3-1bb0-4b78-9b0c-a8439dbdd095 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:45.133477 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5326a8d3-1bb0-4b78-9b0c-a8439dbdd095 None admin] PATH_INFO: `/v3/projects/b817b0f034a64d72914672463eed8f62` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:45.133940 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cdb4ed-aa50-40b3-b134-76edba8fa5df None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:45.134925 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5326a8d3-1bb0-4b78-9b0c-a8439dbdd095 None admin] RBAC: Authorizing `identity:get_project(project_id=b817b0f034a64d72914672463eed8f62)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:45.136509 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5326a8d3-1bb0-4b78-9b0c-a8439dbdd095 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:45.138131 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 186/365] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Sep 1 14:45:44 2022] GET /identity/v3/projects/b817b0f034a64d72914672463eed8f62 => generated 302 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:45:45.138988 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5326a8d3-1bb0-4b78-9b0c-a8439dbdd095 None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:45.172570 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c540270e-cb5b-415f-861f-3df00258066f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:45.173200 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c540270e-cb5b-415f-861f-3df00258066f None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:45.173478 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c540270e-cb5b-415f-861f-3df00258066f None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:45.173879 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c540270e-cb5b-415f-861f-3df00258066f None admin] PATH_INFO: `/v3/users/b4c009c99c384f8ea99b896ad585ce37` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:45.175071 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c540270e-cb5b-415f-861f-3df00258066f None admin] RBAC: Authorizing `identity:get_user(user_id=b4c009c99c384f8ea99b896ad585ce37)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:45.176560 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c540270e-cb5b-415f-861f-3df00258066f None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:45.178894 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 180/366] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Sep 1 14:45:45 2022] GET /identity/v3/users/b4c009c99c384f8ea99b896ad585ce37 => generated 293 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:45:45.184912 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c540270e-cb5b-415f-861f-3df00258066f None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:45.235223 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:45.281452 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5326a8d3-1bb0-4b78-9b0c-a8439dbdd095 None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:45.307379 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c540270e-cb5b-415f-861f-3df00258066f None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:45.322480 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa01b78-6dd4-488b-9f9d-3123320a2ced None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:45.323098 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa01b78-6dd4-488b-9f9d-3123320a2ced None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:45.323379 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa01b78-6dd4-488b-9f9d-3123320a2ced None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:45.323653 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa01b78-6dd4-488b-9f9d-3123320a2ced None admin] PATH_INFO: `/v3/roles` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:45.324317 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa01b78-6dd4-488b-9f9d-3123320a2ced None admin] RBAC: Adding query filter params (name=admin) {{(pid=81342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:45:45.324707 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa01b78-6dd4-488b-9f9d-3123320a2ced None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:45.325990 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa01b78-6dd4-488b-9f9d-3123320a2ced None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:45.334229 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 187/367] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Sep 1 14:45:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:45:45.334911 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa01b78-6dd4-488b-9f9d-3123320a2ced None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:45.348430 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1908f041-18a5-43af-83d0-7926f7f4f1ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:45.349035 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1908f041-18a5-43af-83d0-7926f7f4f1ac None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:45.349838 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1908f041-18a5-43af-83d0-7926f7f4f1ac None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:45.350112 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1908f041-18a5-43af-83d0-7926f7f4f1ac None admin] PATH_INFO: `/v3/roles` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:45.350860 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1908f041-18a5-43af-83d0-7926f7f4f1ac None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81341) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:45:45.351114 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1908f041-18a5-43af-83d0-7926f7f4f1ac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:45.352484 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1908f041-18a5-43af-83d0-7926f7f4f1ac None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:45.360647 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 181/368] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Sep 1 14:45:45 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:45:45.366224 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1908f041-18a5-43af-83d0-7926f7f4f1ac None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:45.395573 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:45.440111 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa01b78-6dd4-488b-9f9d-3123320a2ced None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:45.469938 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1908f041-18a5-43af-83d0-7926f7f4f1ac None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:45.471089 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:45.477379 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc9aae2-15df-4b84-9d0e-8aa3eb59074c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:45.478345 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc9aae2-15df-4b84-9d0e-8aa3eb59074c None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:45.478799 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc9aae2-15df-4b84-9d0e-8aa3eb59074c None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:45.479072 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc9aae2-15df-4b84-9d0e-8aa3eb59074c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:45.479807 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc9aae2-15df-4b84-9d0e-8aa3eb59074c None admin] RBAC: Adding query filter params (group.id=2bca6ad19b2849619ed210863be8b696, role.id=9996407d588d4a00836c8c50e7f37b1a, scope.project.id=1f696f55becc4e879b6d1bac9d93d055) {{(pid=81342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:45:45.480116 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc9aae2-15df-4b84-9d0e-8aa3eb59074c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:45.481563 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc9aae2-15df-4b84-9d0e-8aa3eb59074c None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:45.498267 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 188/369] 10.222.0.33 () {56 vars in 1376 bytes} [Thu Sep 1 14:45:45 2022] GET /identity/v3/role_assignments?group.id=2bca6ad19b2849619ed210863be8b696&scope.project.id=1f696f55becc4e879b6d1bac9d93d055&role.id=9996407d588d4a00836c8c50e7f37b1a => generated 260 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:45:45.498267 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc9aae2-15df-4b84-9d0e-8aa3eb59074c None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:45.519411 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1389485b-38e6-4401-b3c8-02bc6f9308df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:45.520059 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1389485b-38e6-4401-b3c8-02bc6f9308df None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:45.520339 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1389485b-38e6-4401-b3c8-02bc6f9308df None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:45.520610 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1389485b-38e6-4401-b3c8-02bc6f9308df None admin] PATH_INFO: `/v3/groups/2bca6ad19b2849619ed210863be8b696` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:45.521847 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1389485b-38e6-4401-b3c8-02bc6f9308df None admin] RBAC: Authorizing `identity:get_group(group_id=2bca6ad19b2849619ed210863be8b696)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:45.523547 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1389485b-38e6-4401-b3c8-02bc6f9308df None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:45.525348 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 182/370] 10.222.0.33 () {56 vars in 1180 bytes} [Thu Sep 1 14:45:45 2022] GET /identity/v3/groups/2bca6ad19b2849619ed210863be8b696 => generated 226 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:45:45.526161 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1389485b-38e6-4401-b3c8-02bc6f9308df None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:45.628135 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc9aae2-15df-4b84-9d0e-8aa3eb59074c None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:45.654864 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:45.672507 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a778fa2-b12d-4b30-a825-7551e042c0ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:45.673278 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a778fa2-b12d-4b30-a825-7551e042c0ee None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:45.673655 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a778fa2-b12d-4b30-a825-7551e042c0ee None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:45.674042 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a778fa2-b12d-4b30-a825-7551e042c0ee None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:45.674897 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1389485b-38e6-4401-b3c8-02bc6f9308df None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:45.675050 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a778fa2-b12d-4b30-a825-7551e042c0ee None admin] RBAC: Adding query filter params (role.id=869a5d710fb347a195e8f7b2cd3f0750, user.id=b4c009c99c384f8ea99b896ad585ce37, scope.system=all) {{(pid=81342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:45:45.675456 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a778fa2-b12d-4b30-a825-7551e042c0ee None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:45.677089 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a778fa2-b12d-4b30-a825-7551e042c0ee None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:45.694295 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 189/371] 10.222.0.33 () {56 vars in 1308 bytes} [Thu Sep 1 14:45:45 2022] GET /identity/v3/role_assignments?user.id=b4c009c99c384f8ea99b896ad585ce37&scope.system=all&role.id=869a5d710fb347a195e8f7b2cd3f0750 => generated 510 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:45:45.694295 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a778fa2-b12d-4b30-a825-7551e042c0ee None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:45.743532 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7aba3e6-5fe8-4519-9028-24dcc56d1473 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:45.746088 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7aba3e6-5fe8-4519-9028-24dcc56d1473 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:45.746805 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7aba3e6-5fe8-4519-9028-24dcc56d1473 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:45.747500 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7aba3e6-5fe8-4519-9028-24dcc56d1473 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:45.812841 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:45.882192 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7aba3e6-5fe8-4519-9028-24dcc56d1473 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:45.883722 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7aba3e6-5fe8-4519-9028-24dcc56d1473 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:45.892269 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a778fa2-b12d-4b30-a825-7551e042c0ee None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:45.929391 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5377234-3518-45ed-8bae-f8af138e596f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:45.930427 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5377234-3518-45ed-8bae-f8af138e596f None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:45.930919 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5377234-3518-45ed-8bae-f8af138e596f None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:45.931381 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5377234-3518-45ed-8bae-f8af138e596f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:45.993092 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:46.002464 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 183/372] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Sep 1 14:45:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 476 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 14:45:46.003708 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7aba3e6-5fe8-4519-9028-24dcc56d1473 None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:46.023792 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5377234-3518-45ed-8bae-f8af138e596f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:46.025294 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5377234-3518-45ed-8bae-f8af138e596f None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:46.098199 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:46.122632 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7aba3e6-5fe8-4519-9028-24dcc56d1473 None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:46.158952 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 190/373] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Sep 1 14:45:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 468 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 14:45:46.159767 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5377234-3518-45ed-8bae-f8af138e596f None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:46.171167 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f10a13-eff0-4c28-ac8b-c218e879085e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:46.172146 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f10a13-eff0-4c28-ac8b-c218e879085e None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:46.172636 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f10a13-eff0-4c28-ac8b-c218e879085e None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:46.173095 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f10a13-eff0-4c28-ac8b-c218e879085e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:46.174017 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f10a13-eff0-4c28-ac8b-c218e879085e None admin] RBAC: Adding query filter params (group.id=2bca6ad19b2849619ed210863be8b696, role.id=5d1feef0e13d4874928bd86258c4e3b2, scope.project.id=b817b0f034a64d72914672463eed8f62) {{(pid=81341) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:45:46.174596 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f10a13-eff0-4c28-ac8b-c218e879085e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:46.176277 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f10a13-eff0-4c28-ac8b-c218e879085e None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:46.182358 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 184/374] 10.222.0.33 () {56 vars in 1376 bytes} [Thu Sep 1 14:45:46 2022] GET /identity/v3/role_assignments?group.id=2bca6ad19b2849619ed210863be8b696&scope.project.id=b817b0f034a64d72914672463eed8f62&role.id=5d1feef0e13d4874928bd86258c4e3b2 => generated 260 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:45:46.183577 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f10a13-eff0-4c28-ac8b-c218e879085e None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:46.261683 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:46.303830 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5377234-3518-45ed-8bae-f8af138e596f None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:46.309057 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f10a13-eff0-4c28-ac8b-c218e879085e None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:46.344219 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18986f7-e765-460d-a3a9-c34a64d6cdce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:46.344960 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18986f7-e765-460d-a3a9-c34a64d6cdce None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:46.345254 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18986f7-e765-460d-a3a9-c34a64d6cdce None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:46.345528 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18986f7-e765-460d-a3a9-c34a64d6cdce None admin] PATH_INFO: `/v3/projects/5cec9ba4a51d40d1be462a47013baf0a` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:46.346739 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18986f7-e765-460d-a3a9-c34a64d6cdce None admin] RBAC: Authorizing `identity:get_project(project_id=5cec9ba4a51d40d1be462a47013baf0a)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:46.347825 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3aac4ba-05cd-4ae5-943d-b9a5ee0acb1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:46.348317 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18986f7-e765-460d-a3a9-c34a64d6cdce None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:46.348883 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3aac4ba-05cd-4ae5-943d-b9a5ee0acb1b None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:46.349399 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3aac4ba-05cd-4ae5-943d-b9a5ee0acb1b None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:46.349870 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3aac4ba-05cd-4ae5-943d-b9a5ee0acb1b None admin] PATH_INFO: `/v3/projects/b817b0f034a64d72914672463eed8f62` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:46.350377 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 191/375] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Sep 1 14:45:46 2022] GET /identity/v3/projects/5cec9ba4a51d40d1be462a47013baf0a => generated 344 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:45:46.355426 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3aac4ba-05cd-4ae5-943d-b9a5ee0acb1b None admin] RBAC: Authorizing `identity:get_project(project_id=b817b0f034a64d72914672463eed8f62)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:46.355560 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18986f7-e765-460d-a3a9-c34a64d6cdce None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:46.357352 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3aac4ba-05cd-4ae5-943d-b9a5ee0acb1b None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:46.359321 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 185/376] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Sep 1 14:45:46 2022] GET /identity/v3/projects/b817b0f034a64d72914672463eed8f62 => generated 302 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:45:46.366150 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3aac4ba-05cd-4ae5-943d-b9a5ee0acb1b None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:46.385239 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:46.473502 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18986f7-e765-460d-a3a9-c34a64d6cdce None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:46.480124 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3aac4ba-05cd-4ae5-943d-b9a5ee0acb1b None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:46.510846 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:46.512373 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23be96a6-72c8-47d9-8d74-126dc10b4f96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:46.512991 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23be96a6-72c8-47d9-8d74-126dc10b4f96 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:46.513270 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23be96a6-72c8-47d9-8d74-126dc10b4f96 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:46.513615 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23be96a6-72c8-47d9-8d74-126dc10b4f96 None admin] PATH_INFO: `/v3/groups/a7d3f001853a464481317bfb0ee59f2f` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:46.520508 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23be96a6-72c8-47d9-8d74-126dc10b4f96 None admin] RBAC: Authorizing `identity:get_group(group_id=a7d3f001853a464481317bfb0ee59f2f)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:46.522015 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23be96a6-72c8-47d9-8d74-126dc10b4f96 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:46.522755 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637a73f2-0a8c-4354-bb55-3a5701f5541f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:46.523885 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637a73f2-0a8c-4354-bb55-3a5701f5541f None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:46.523993 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 192/377] 10.222.0.33 () {56 vars in 1180 bytes} [Thu Sep 1 14:45:46 2022] GET /identity/v3/groups/a7d3f001853a464481317bfb0ee59f2f => generated 229 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 14:45:46.524501 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637a73f2-0a8c-4354-bb55-3a5701f5541f None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:46.525033 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637a73f2-0a8c-4354-bb55-3a5701f5541f None admin] PATH_INFO: `/v3/groups/2bca6ad19b2849619ed210863be8b696` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:46.526465 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637a73f2-0a8c-4354-bb55-3a5701f5541f None admin] RBAC: Authorizing `identity:get_group(group_id=2bca6ad19b2849619ed210863be8b696)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:46.528206 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637a73f2-0a8c-4354-bb55-3a5701f5541f None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:46.529482 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23be96a6-72c8-47d9-8d74-126dc10b4f96 None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:46.530438 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 186/378] 10.222.0.33 () {56 vars in 1180 bytes} [Thu Sep 1 14:45:46 2022] GET /identity/v3/groups/2bca6ad19b2849619ed210863be8b696 => generated 226 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 14:45:46.536782 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637a73f2-0a8c-4354-bb55-3a5701f5541f None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:46.624675 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:46.641732 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23be96a6-72c8-47d9-8d74-126dc10b4f96 None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:46.667165 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637a73f2-0a8c-4354-bb55-3a5701f5541f None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:46.684563 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33e3388-519a-4d17-a3ba-81bbc7e13066 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:46.685270 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33e3388-519a-4d17-a3ba-81bbc7e13066 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:46.685557 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33e3388-519a-4d17-a3ba-81bbc7e13066 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:46.685829 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33e3388-519a-4d17-a3ba-81bbc7e13066 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:46.686596 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33e3388-519a-4d17-a3ba-81bbc7e13066 None admin] RBAC: Adding query filter params (group.id=a7d3f001853a464481317bfb0ee59f2f, role.id=2a2adb769e55476b82e5014897e0a9c0, scope.project.id=5cec9ba4a51d40d1be462a47013baf0a) {{(pid=81342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:45:46.686902 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33e3388-519a-4d17-a3ba-81bbc7e13066 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:46.688437 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33e3388-519a-4d17-a3ba-81bbc7e13066 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:46.695355 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 193/379] 10.222.0.33 () {56 vars in 1376 bytes} [Thu Sep 1 14:45:46 2022] GET /identity/v3/role_assignments?group.id=a7d3f001853a464481317bfb0ee59f2f&scope.project.id=5cec9ba4a51d40d1be462a47013baf0a&role.id=2a2adb769e55476b82e5014897e0a9c0 => generated 260 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 14:45:46.696826 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f033009-d6ed-45b1-987e-17e148057e05 None None] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:46.697225 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f033009-d6ed-45b1-987e-17e148057e05 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:46.697508 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f033009-d6ed-45b1-987e-17e148057e05 None None] PATH_INFO: `/` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:46.707249 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 194/380] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 1 14:45:46 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:45:46.710989 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9496f5-6455-4404-b9da-5e7a24986622 None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:46.711273 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9496f5-6455-4404-b9da-5e7a24986622 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:46.711551 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9496f5-6455-4404-b9da-5e7a24986622 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:46.725212 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b92778-c7af-4538-93c0-59751796296f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:46.726155 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b92778-c7af-4538-93c0-59751796296f None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:46.728697 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b92778-c7af-4538-93c0-59751796296f None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:46.729204 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b92778-c7af-4538-93c0-59751796296f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:46.731131 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b92778-c7af-4538-93c0-59751796296f None admin] RBAC: Adding query filter params (group.id=2bca6ad19b2849619ed210863be8b696, role.id=9996407d588d4a00836c8c50e7f37b1a, scope.project.id=b817b0f034a64d72914672463eed8f62) {{(pid=81341) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:45:46.731641 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b92778-c7af-4538-93c0-59751796296f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:46.736541 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b92778-c7af-4538-93c0-59751796296f None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:46.749136 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 187/381] 10.222.0.33 () {56 vars in 1376 bytes} [Thu Sep 1 14:45:46 2022] GET /identity/v3/role_assignments?group.id=2bca6ad19b2849619ed210863be8b696&scope.project.id=b817b0f034a64d72914672463eed8f62&role.id=9996407d588d4a00836c8c50e7f37b1a => generated 260 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 14:45:46.750935 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-0b9496f5-6455-4404-b9da-5e7a24986622 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:45:46.752382 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-0b9496f5-6455-4404-b9da-5e7a24986622 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:45:46.832828 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 195/382] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 1 14:45:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 14:45:47.129697 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b92778-c7af-4538-93c0-59751796296f None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:47.215233 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:47.266324 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b92778-c7af-4538-93c0-59751796296f None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:47.315126 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8221e182-fe07-434b-824e-341f9a88476a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:47.315767 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8221e182-fe07-434b-824e-341f9a88476a None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:47.316048 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8221e182-fe07-434b-824e-341f9a88476a None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:47.316321 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8221e182-fe07-434b-824e-341f9a88476a None admin] PATH_INFO: `/v3/roles/member` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:47.331292 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8221e182-fe07-434b-824e-341f9a88476a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 14:45:47.337460 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application [None req-8221e182-fe07-434b-824e-341f9a88476a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 14:45:47.337460 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:45:47.337460 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:45:47.337460 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:45:47.337460 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:45:47.337460 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:45:47.337460 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:45:47.337460 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:45:47.337460 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:45:47.337460 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:45:47.337460 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:45:47.337460 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:45:47.337460 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 14:45:47.337460 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 14:45:47.337460 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 14:45:47.337460 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 14:45:47.338390 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 14:45:47.338390 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 14:45:47.338390 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 14:45:47.338390 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 14:45:47.338390 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:45:47.338390 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:45:47.338390 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:45:47.338390 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:45:47.338390 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:45:47.338390 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:45:47.338390 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:45:47.338390 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application with Lock( Sep 01 14:45:47.338390 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:45:47.338390 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:45:47.338390 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:45:47.338390 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:45:47.339200 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:45:47.339200 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:45:47.339200 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:45:47.339200 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:45:47.339200 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 14:45:47.339200 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 14:45:47.339200 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 14:45:47.339200 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 14:45:47.339200 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 14:45:47.339200 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 14:45:47.339200 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 14:45:47.339200 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Sep 01 14:45:47.342268 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 188/383] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Sep 1 14:45:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:45:47.347128 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9496f5-6455-4404-b9da-5e7a24986622 None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:47.447093 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:47.474084 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9496f5-6455-4404-b9da-5e7a24986622 None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:47.519673 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2757bc45-6724-4993-b2d9-7206c62f19fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:47.520611 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2757bc45-6724-4993-b2d9-7206c62f19fb None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:47.521626 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2757bc45-6724-4993-b2d9-7206c62f19fb None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:47.522088 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2757bc45-6724-4993-b2d9-7206c62f19fb None admin] PATH_INFO: `/v3/roles` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:47.522959 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2757bc45-6724-4993-b2d9-7206c62f19fb None admin] RBAC: Adding query filter params (name=member) {{(pid=81342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:45:47.523463 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2757bc45-6724-4993-b2d9-7206c62f19fb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:47.525023 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2757bc45-6724-4993-b2d9-7206c62f19fb None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:47.538058 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 196/384] 10.222.0.33 () {56 vars in 1101 bytes} [Thu Sep 1 14:45:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:45:47.543305 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8221e182-fe07-434b-824e-341f9a88476a None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:47.700153 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8221e182-fe07-434b-824e-341f9a88476a None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:47.730172 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:47.747074 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258d5d2b-8406-419d-8c16-9ea0d930a27b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:47.747716 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258d5d2b-8406-419d-8c16-9ea0d930a27b None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:47.747997 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258d5d2b-8406-419d-8c16-9ea0d930a27b None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:47.748267 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258d5d2b-8406-419d-8c16-9ea0d930a27b None admin] PATH_INFO: `/v3/groups/2bca6ad19b2849619ed210863be8b696` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:47.754929 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258d5d2b-8406-419d-8c16-9ea0d930a27b None admin] RBAC: Authorizing `identity:get_group(group_id=2bca6ad19b2849619ed210863be8b696)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:47.756932 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258d5d2b-8406-419d-8c16-9ea0d930a27b None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:47.758754 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 189/385] 10.222.0.33 () {56 vars in 1180 bytes} [Thu Sep 1 14:45:47 2022] GET /identity/v3/groups/2bca6ad19b2849619ed210863be8b696 => generated 226 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 14:45:47.764965 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f533cf-2f5b-4d55-9530-4edc27513cc4 None None] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:47.764965 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f533cf-2f5b-4d55-9530-4edc27513cc4 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:47.764965 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f533cf-2f5b-4d55-9530-4edc27513cc4 None None] PATH_INFO: `/` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:47.765216 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258d5d2b-8406-419d-8c16-9ea0d930a27b None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:47.765310 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 197/386] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 1 14:45:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:45:47.770176 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5078345e-2e53-444b-85c4-8e993b0c26af None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:47.770728 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5078345e-2e53-444b-85c4-8e993b0c26af None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:47.771074 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5078345e-2e53-444b-85c4-8e993b0c26af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:47.807218 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-5078345e-2e53-444b-85c4-8e993b0c26af None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:45:47.808439 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-5078345e-2e53-444b-85c4-8e993b0c26af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:45:47.905043 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258d5d2b-8406-419d-8c16-9ea0d930a27b None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:47.935893 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 198/387] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 1 14:45:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 14:45:47.952584 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f6184e-d362-4fb1-b11d-04be345359e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:47.953491 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f6184e-d362-4fb1-b11d-04be345359e4 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:47.953921 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f6184e-d362-4fb1-b11d-04be345359e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:47.954371 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f6184e-d362-4fb1-b11d-04be345359e4 None admin] PATH_INFO: `/v3/projects/1f696f55becc4e879b6d1bac9d93d055` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:47.955688 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f6184e-d362-4fb1-b11d-04be345359e4 None admin] RBAC: Authorizing `identity:get_project(project_id=1f696f55becc4e879b6d1bac9d93d055)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:47.957284 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f6184e-d362-4fb1-b11d-04be345359e4 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:47.959892 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 190/388] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Sep 1 14:45:47 2022] GET /identity/v3/projects/1f696f55becc4e879b6d1bac9d93d055 => generated 298 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 14:45:47.965424 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5078345e-2e53-444b-85c4-8e993b0c26af None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:48.087340 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:48.128974 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5078345e-2e53-444b-85c4-8e993b0c26af None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:48.179654 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b520b21-6cd7-40cf-bef0-d09d659c927f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:48.180375 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b520b21-6cd7-40cf-bef0-d09d659c927f None admin] REQUEST_METHOD: `PUT` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:48.180653 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b520b21-6cd7-40cf-bef0-d09d659c927f None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:48.180919 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b520b21-6cd7-40cf-bef0-d09d659c927f None admin] PATH_INFO: `/v3/projects/1f696f55becc4e879b6d1bac9d93d055/groups/2bca6ad19b2849619ed210863be8b696/roles/5d1feef0e13d4874928bd86258c4e3b2` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:48.187555 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b520b21-6cd7-40cf-bef0-d09d659c927f None admin] RBAC: Authorizing `identity:create_grant(project_id=1f696f55becc4e879b6d1bac9d93d055, group_id=2bca6ad19b2849619ed210863be8b696, role_id=5d1feef0e13d4874928bd86258c4e3b2)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:48.188999 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b520b21-6cd7-40cf-bef0-d09d659c927f None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:48.213853 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 199/389] 10.222.0.33 () {58 vars in 1442 bytes} [Thu Sep 1 14:45:47 2022] PUT /identity/v3/projects/1f696f55becc4e879b6d1bac9d93d055/groups/2bca6ad19b2849619ed210863be8b696/roles/5d1feef0e13d4874928bd86258c4e3b2 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:45:48.308388 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f6184e-d362-4fb1-b11d-04be345359e4 None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:48.387266 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:48.445493 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f6184e-d362-4fb1-b11d-04be345359e4 None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:48.489789 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5cc25f-359b-4603-ab92-26186dc781b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:48.490875 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5cc25f-359b-4603-ab92-26186dc781b1 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:48.491156 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5cc25f-359b-4603-ab92-26186dc781b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:48.491428 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5cc25f-359b-4603-ab92-26186dc781b1 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:48.508090 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5cc25f-359b-4603-ab92-26186dc781b1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 14:45:48.514539 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application [None req-7a5cc25f-359b-4603-ab92-26186dc781b1 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 14:45:48.514539 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:45:48.514539 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:45:48.514539 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:45:48.514539 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:45:48.514539 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:45:48.514539 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:45:48.514539 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:45:48.514539 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:45:48.514539 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:45:48.514539 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:45:48.514539 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:45:48.514539 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 14:45:48.514539 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 14:45:48.514539 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 14:45:48.514539 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 14:45:48.515477 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 14:45:48.515477 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 14:45:48.515477 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 14:45:48.515477 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 14:45:48.515477 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:45:48.515477 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:45:48.515477 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:45:48.515477 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:45:48.515477 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:45:48.515477 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:45:48.515477 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:45:48.515477 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application with Lock( Sep 01 14:45:48.515477 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:45:48.515477 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:45:48.515477 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:45:48.515477 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:45:48.516357 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:45:48.516357 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:45:48.516357 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:45:48.516357 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:45:48.516357 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 14:45:48.516357 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 14:45:48.516357 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 14:45:48.516357 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 14:45:48.516357 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 14:45:48.516357 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 14:45:48.516357 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 14:45:48.516357 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Sep 01 14:45:48.516357 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 191/390] 10.222.0.33 () {56 vars in 1114 bytes} [Thu Sep 1 14:45:48 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:45:48.521985 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b520b21-6cd7-40cf-bef0-d09d659c927f None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:48.633079 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b520b21-6cd7-40cf-bef0-d09d659c927f None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:48.670261 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:48.677925 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3859bd08-155a-431c-973e-10e2d1ada3f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:48.678641 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3859bd08-155a-431c-973e-10e2d1ada3f7 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:48.678914 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3859bd08-155a-431c-973e-10e2d1ada3f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:48.679174 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3859bd08-155a-431c-973e-10e2d1ada3f7 None admin] PATH_INFO: `/v3/roles` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:48.679843 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3859bd08-155a-431c-973e-10e2d1ada3f7 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:45:48.680137 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3859bd08-155a-431c-973e-10e2d1ada3f7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:48.681521 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3859bd08-155a-431c-973e-10e2d1ada3f7 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:48.696892 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 200/391] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Sep 1 14:45:48 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:45:48.703383 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5cc25f-359b-4603-ab92-26186dc781b1 None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:48.843629 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5cc25f-359b-4603-ab92-26186dc781b1 None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:48.867035 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:48.885672 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0644de7c-2624-4709-804d-6f4f80e604c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:48.886333 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0644de7c-2624-4709-804d-6f4f80e604c3 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:48.886611 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0644de7c-2624-4709-804d-6f4f80e604c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:48.886883 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0644de7c-2624-4709-804d-6f4f80e604c3 None admin] PATH_INFO: `/v3/groups/2bca6ad19b2849619ed210863be8b696` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:48.888653 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0644de7c-2624-4709-804d-6f4f80e604c3 None admin] RBAC: Authorizing `identity:get_group(group_id=2bca6ad19b2849619ed210863be8b696)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:48.889794 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e09e8e-f674-430c-86bc-f32e4c4ebe57 None None] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:48.890120 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0644de7c-2624-4709-804d-6f4f80e604c3 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:48.890611 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e09e8e-f674-430c-86bc-f32e4c4ebe57 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:48.891084 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e09e8e-f674-430c-86bc-f32e4c4ebe57 None None] PATH_INFO: `/` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:48.892022 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 192/392] 10.222.0.33 () {56 vars in 1180 bytes} [Thu Sep 1 14:45:48 2022] GET /identity/v3/groups/2bca6ad19b2849619ed210863be8b696 => generated 226 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:45:48.892814 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 201/393] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 1 14:45:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 14:45:48.897926 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0644de7c-2624-4709-804d-6f4f80e604c3 None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:48.899897 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0419c6f2-2553-4fc1-b6c4-5a37d8bc5955 None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:48.900389 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0419c6f2-2553-4fc1-b6c4-5a37d8bc5955 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:48.900857 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0419c6f2-2553-4fc1-b6c4-5a37d8bc5955 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:48.934517 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-0419c6f2-2553-4fc1-b6c4-5a37d8bc5955 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:45:48.936056 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-0419c6f2-2553-4fc1-b6c4-5a37d8bc5955 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:45:49.025514 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0644de7c-2624-4709-804d-6f4f80e604c3 None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:49.075249 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 202/394] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 1 14:45:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 14:45:49.076611 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a60cbbd-d9e2-430b-856f-b71b30c43dec None None] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:49.076907 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a60cbbd-d9e2-430b-856f-b71b30c43dec None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:49.077181 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a60cbbd-d9e2-430b-856f-b71b30c43dec None None] PATH_INFO: `/` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:49.078711 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 203/395] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 1 14:45:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:45:49.079347 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e577f8ca-9c87-4364-85a4-1f3d91576da5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:49.080198 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e577f8ca-9c87-4364-85a4-1f3d91576da5 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:49.080672 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e577f8ca-9c87-4364-85a4-1f3d91576da5 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:49.081156 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e577f8ca-9c87-4364-85a4-1f3d91576da5 None admin] PATH_INFO: `/v3/projects/1f696f55becc4e879b6d1bac9d93d055` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:49.082679 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e577f8ca-9c87-4364-85a4-1f3d91576da5 None admin] RBAC: Authorizing `identity:get_project(project_id=1f696f55becc4e879b6d1bac9d93d055)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:49.084335 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb354ae-7ad3-4d18-a018-a1163aa811b6 None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:49.084620 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb354ae-7ad3-4d18-a018-a1163aa811b6 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:49.084894 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb354ae-7ad3-4d18-a018-a1163aa811b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:49.085635 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e577f8ca-9c87-4364-85a4-1f3d91576da5 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:49.087916 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 193/396] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Sep 1 14:45:48 2022] GET /identity/v3/projects/1f696f55becc4e879b6d1bac9d93d055 => generated 298 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:45:49.099620 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e577f8ca-9c87-4364-85a4-1f3d91576da5 None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:49.103538 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-2cb354ae-7ad3-4d18-a018-a1163aa811b6 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:45:49.105645 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-2cb354ae-7ad3-4d18-a018-a1163aa811b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:45:49.183588 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 204/397] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 1 14:45:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 14:45:49.184146 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:49.185832 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92aa095-a3d1-44fe-8373-07e6e1c9bcd3 None None] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:49.186117 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92aa095-a3d1-44fe-8373-07e6e1c9bcd3 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:49.186509 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92aa095-a3d1-44fe-8373-07e6e1c9bcd3 None None] PATH_INFO: `/` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:49.188045 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 205/398] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 1 14:45:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:45:49.195274 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1145356-e82f-48f4-a8ad-968fd49b5341 None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:49.195561 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1145356-e82f-48f4-a8ad-968fd49b5341 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:49.196105 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1145356-e82f-48f4-a8ad-968fd49b5341 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:49.235351 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-c1145356-e82f-48f4-a8ad-968fd49b5341 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:45:49.236606 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-c1145356-e82f-48f4-a8ad-968fd49b5341 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:45:49.247945 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e577f8ca-9c87-4364-85a4-1f3d91576da5 None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:49.295808 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f537b36a-5adb-4a26-9944-331e673ce4ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:49.296466 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f537b36a-5adb-4a26-9944-331e673ce4ac None admin] REQUEST_METHOD: `PUT` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:49.296744 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f537b36a-5adb-4a26-9944-331e673ce4ac None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:49.297015 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f537b36a-5adb-4a26-9944-331e673ce4ac None admin] PATH_INFO: `/v3/projects/1f696f55becc4e879b6d1bac9d93d055/groups/2bca6ad19b2849619ed210863be8b696/roles/9996407d588d4a00836c8c50e7f37b1a` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:49.301976 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f537b36a-5adb-4a26-9944-331e673ce4ac None admin] RBAC: Authorizing `identity:create_grant(project_id=1f696f55becc4e879b6d1bac9d93d055, group_id=2bca6ad19b2849619ed210863be8b696, role_id=9996407d588d4a00836c8c50e7f37b1a)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:49.303472 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f537b36a-5adb-4a26-9944-331e673ce4ac None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:49.305675 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:49.325388 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 206/399] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 1 14:45:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 14:45:49.331932 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 194/400] 10.222.0.33 () {58 vars in 1442 bytes} [Thu Sep 1 14:45:49 2022] PUT /identity/v3/projects/1f696f55becc4e879b6d1bac9d93d055/groups/2bca6ad19b2849619ed210863be8b696/roles/9996407d588d4a00836c8c50e7f37b1a => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:45:49.347657 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1145356-e82f-48f4-a8ad-968fd49b5341 None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:49.485375 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1145356-e82f-48f4-a8ad-968fd49b5341 None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:49.536887 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7000f2f9-4705-4cc5-9c6d-174c81f48db4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:49.537540 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7000f2f9-4705-4cc5-9c6d-174c81f48db4 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:49.537780 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7000f2f9-4705-4cc5-9c6d-174c81f48db4 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:49.538047 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7000f2f9-4705-4cc5-9c6d-174c81f48db4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:49.544638 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f537b36a-5adb-4a26-9944-331e673ce4ac None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:49.549850 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7000f2f9-4705-4cc5-9c6d-174c81f48db4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 14:45:49.555238 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application [None req-7000f2f9-4705-4cc5-9c6d-174c81f48db4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 14:45:49.555238 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:45:49.555238 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:45:49.555238 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:45:49.555238 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:45:49.555238 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:45:49.555238 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:45:49.555238 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:45:49.555238 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:45:49.555238 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:45:49.555238 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:45:49.555238 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:45:49.555238 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 14:45:49.555238 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 14:45:49.555238 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 14:45:49.555238 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 14:45:49.556217 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 14:45:49.556217 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 14:45:49.556217 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 14:45:49.556217 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 14:45:49.556217 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:45:49.556217 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:45:49.556217 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:45:49.556217 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:45:49.556217 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:45:49.556217 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:45:49.556217 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:45:49.556217 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application with Lock( Sep 01 14:45:49.556217 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:45:49.556217 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:45:49.556217 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:45:49.556217 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:45:49.557061 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:45:49.557061 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:45:49.557061 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:45:49.557061 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:45:49.557061 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 14:45:49.557061 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 14:45:49.557061 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 14:45:49.557061 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 14:45:49.557061 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 14:45:49.557061 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 14:45:49.557061 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 14:45:49.557061 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Sep 01 14:45:49.557061 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 207/401] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Sep 1 14:45:49 2022] GET /identity/v3/roles/member => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:45:49.562466 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7000f2f9-4705-4cc5-9c6d-174c81f48db4 None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:49.627393 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:49.702302 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f537b36a-5adb-4a26-9944-331e673ce4ac None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:49.708452 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7000f2f9-4705-4cc5-9c6d-174c81f48db4 None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:49.715741 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:49.738358 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7ecabb-d348-4974-98b7-31aa0f6e240c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:49.738907 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7ecabb-d348-4974-98b7-31aa0f6e240c None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:49.739258 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7ecabb-d348-4974-98b7-31aa0f6e240c None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:49.739596 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7ecabb-d348-4974-98b7-31aa0f6e240c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:49.747712 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1997328b-58a8-4a0c-ad97-87850239a1ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:49.748587 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1997328b-58a8-4a0c-ad97-87850239a1ec None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:49.749057 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1997328b-58a8-4a0c-ad97-87850239a1ec None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:49.749541 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1997328b-58a8-4a0c-ad97-87850239a1ec None admin] PATH_INFO: `/v3/roles` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:49.750479 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1997328b-58a8-4a0c-ad97-87850239a1ec None admin] RBAC: Adding query filter params (name=member) {{(pid=81342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:45:49.751012 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1997328b-58a8-4a0c-ad97-87850239a1ec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:49.752403 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7ecabb-d348-4974-98b7-31aa0f6e240c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 14:45:49.752835 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1997328b-58a8-4a0c-ad97-87850239a1ec None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:49.758698 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application [None req-6e7ecabb-d348-4974-98b7-31aa0f6e240c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 14:45:49.758698 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:45:49.758698 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:45:49.758698 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:45:49.758698 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:45:49.758698 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:45:49.758698 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:45:49.758698 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:45:49.758698 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:45:49.758698 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:45:49.758698 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:45:49.758698 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:45:49.758698 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 14:45:49.758698 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 14:45:49.758698 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 14:45:49.758698 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 14:45:49.759627 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 14:45:49.759627 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 14:45:49.759627 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 14:45:49.759627 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 14:45:49.759627 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:45:49.759627 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:45:49.759627 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:45:49.759627 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:45:49.759627 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:45:49.759627 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:45:49.759627 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:45:49.759627 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application with Lock( Sep 01 14:45:49.759627 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:45:49.759627 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:45:49.759627 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:45:49.759627 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:45:49.760441 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:45:49.760441 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:45:49.760441 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:45:49.760441 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:45:49.760441 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 14:45:49.760441 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 14:45:49.760441 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 14:45:49.760441 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 14:45:49.760441 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 14:45:49.760441 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 14:45:49.760441 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 14:45:49.760441 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Sep 01 14:45:49.760441 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 195/402] 10.222.0.33 () {56 vars in 1114 bytes} [Thu Sep 1 14:45:49 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:45:49.761663 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7ecabb-d348-4974-98b7-31aa0f6e240c None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:49.766437 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 208/403] 10.222.0.33 () {56 vars in 1101 bytes} [Thu Sep 1 14:45:49 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 14:45:49.767603 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1997328b-58a8-4a0c-ad97-87850239a1ec None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:49.881418 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1997328b-58a8-4a0c-ad97-87850239a1ec None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:49.889449 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:49.902600 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7ecabb-d348-4974-98b7-31aa0f6e240c None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:49.922222 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707bd5ef-63a0-43cd-89f6-af3c44e11164 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:49.923339 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707bd5ef-63a0-43cd-89f6-af3c44e11164 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:49.924105 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707bd5ef-63a0-43cd-89f6-af3c44e11164 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:49.924583 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707bd5ef-63a0-43cd-89f6-af3c44e11164 None admin] PATH_INFO: `/v3/roles` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:49.925577 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707bd5ef-63a0-43cd-89f6-af3c44e11164 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:45:49.926098 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707bd5ef-63a0-43cd-89f6-af3c44e11164 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:49.928991 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707bd5ef-63a0-43cd-89f6-af3c44e11164 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:49.940334 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 209/404] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Sep 1 14:45:49 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:45:49.941502 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707bd5ef-63a0-43cd-89f6-af3c44e11164 None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:49.942658 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa7cca3-fc64-4fc9-bded-46c37c679380 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:49.943327 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa7cca3-fc64-4fc9-bded-46c37c679380 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:49.943600 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa7cca3-fc64-4fc9-bded-46c37c679380 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:49.943867 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa7cca3-fc64-4fc9-bded-46c37c679380 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:49.956931 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa7cca3-fc64-4fc9-bded-46c37c679380 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 14:45:49.965591 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application [None req-0aa7cca3-fc64-4fc9-bded-46c37c679380 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 14:45:49.965591 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:45:49.965591 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:45:49.965591 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:45:49.965591 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:45:49.965591 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:45:49.965591 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:45:49.965591 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:45:49.965591 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:45:49.965591 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:45:49.965591 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:45:49.965591 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:45:49.965591 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 14:45:49.965591 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 14:45:49.965591 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 14:45:49.965591 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 14:45:49.966542 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 14:45:49.966542 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 14:45:49.966542 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 14:45:49.966542 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 14:45:49.966542 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:45:49.966542 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:45:49.966542 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:45:49.966542 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:45:49.966542 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:45:49.966542 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:45:49.966542 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:45:49.966542 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application with Lock( Sep 01 14:45:49.966542 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:45:49.966542 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:45:49.966542 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:45:49.966542 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:45:49.967383 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:45:49.967383 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:45:49.967383 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:45:49.967383 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:45:49.967383 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 14:45:49.967383 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 14:45:49.967383 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 14:45:49.967383 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 14:45:49.967383 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 14:45:49.967383 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 14:45:49.967383 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 14:45:49.967383 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Sep 01 14:45:49.970310 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 196/405] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Sep 1 14:45:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:45:49.971287 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa7cca3-fc64-4fc9-bded-46c37c679380 None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:50.007700 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:50.054701 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707bd5ef-63a0-43cd-89f6-af3c44e11164 None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:50.083533 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa7cca3-fc64-4fc9-bded-46c37c679380 None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:50.088438 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7092fc1f-4103-4be3-be60-5528f241540c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:50.089063 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7092fc1f-4103-4be3-be60-5528f241540c None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:50.089342 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7092fc1f-4103-4be3-be60-5528f241540c None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:50.089611 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7092fc1f-4103-4be3-be60-5528f241540c None admin] PATH_INFO: `/v3/groups/2bca6ad19b2849619ed210863be8b696` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:50.090756 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7092fc1f-4103-4be3-be60-5528f241540c None admin] RBAC: Authorizing `identity:get_group(group_id=2bca6ad19b2849619ed210863be8b696)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:50.092310 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7092fc1f-4103-4be3-be60-5528f241540c None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:50.098647 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 210/406] 10.222.0.33 () {56 vars in 1180 bytes} [Thu Sep 1 14:45:49 2022] GET /identity/v3/groups/2bca6ad19b2849619ed210863be8b696 => generated 226 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:45:50.100021 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7092fc1f-4103-4be3-be60-5528f241540c None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:50.127267 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f09c5f-fce2-4097-ae48-a1f9d089a479 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:50.128263 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f09c5f-fce2-4097-ae48-a1f9d089a479 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:50.128847 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f09c5f-fce2-4097-ae48-a1f9d089a479 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:50.129271 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f09c5f-fce2-4097-ae48-a1f9d089a479 None admin] PATH_INFO: `/v3/groups/2bca6ad19b2849619ed210863be8b696` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:50.131264 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f09c5f-fce2-4097-ae48-a1f9d089a479 None admin] RBAC: Authorizing `identity:get_group(group_id=2bca6ad19b2849619ed210863be8b696)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:50.133859 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f09c5f-fce2-4097-ae48-a1f9d089a479 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:50.136697 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 197/407] 10.222.0.33 () {56 vars in 1180 bytes} [Thu Sep 1 14:45:49 2022] GET /identity/v3/groups/2bca6ad19b2849619ed210863be8b696 => generated 226 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:45:50.137721 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f09c5f-fce2-4097-ae48-a1f9d089a479 None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:50.140829 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:50.226294 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7092fc1f-4103-4be3-be60-5528f241540c None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:50.265681 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:50.267788 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14620841-17ee-42d0-8023-9c62f1d0d269 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:50.270364 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14620841-17ee-42d0-8023-9c62f1d0d269 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:50.270364 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14620841-17ee-42d0-8023-9c62f1d0d269 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:50.270364 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14620841-17ee-42d0-8023-9c62f1d0d269 None admin] PATH_INFO: `/v3/roles` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:50.270649 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f09c5f-fce2-4097-ae48-a1f9d089a479 None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:50.270744 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14620841-17ee-42d0-8023-9c62f1d0d269 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:45:50.270744 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14620841-17ee-42d0-8023-9c62f1d0d269 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:50.273005 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14620841-17ee-42d0-8023-9c62f1d0d269 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:50.284197 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 211/408] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Sep 1 14:45:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:45:50.285208 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14620841-17ee-42d0-8023-9c62f1d0d269 None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:50.312287 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6b91eb-4184-43ea-8323-b46e653a6730 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:50.313001 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6b91eb-4184-43ea-8323-b46e653a6730 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:50.313441 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6b91eb-4184-43ea-8323-b46e653a6730 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:50.313722 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6b91eb-4184-43ea-8323-b46e653a6730 None admin] PATH_INFO: `/v3/projects/b817b0f034a64d72914672463eed8f62` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:50.314930 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6b91eb-4184-43ea-8323-b46e653a6730 None admin] RBAC: Authorizing `identity:get_project(project_id=b817b0f034a64d72914672463eed8f62)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:50.316734 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6b91eb-4184-43ea-8323-b46e653a6730 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:50.318654 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 198/409] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Sep 1 14:45:50 2022] GET /identity/v3/projects/b817b0f034a64d72914672463eed8f62 => generated 302 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:45:50.324440 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6b91eb-4184-43ea-8323-b46e653a6730 None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:50.408575 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:50.415482 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14620841-17ee-42d0-8023-9c62f1d0d269 None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:50.433081 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6b91eb-4184-43ea-8323-b46e653a6730 None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:50.460005 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6368c311-b98b-4a4c-9a3e-659e2dca8c82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:50.461064 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6368c311-b98b-4a4c-9a3e-659e2dca8c82 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:50.461812 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6368c311-b98b-4a4c-9a3e-659e2dca8c82 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:50.462269 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6368c311-b98b-4a4c-9a3e-659e2dca8c82 None admin] PATH_INFO: `/v3/projects/b817b0f034a64d72914672463eed8f62` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:50.463971 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6368c311-b98b-4a4c-9a3e-659e2dca8c82 None admin] RBAC: Authorizing `identity:get_project(project_id=b817b0f034a64d72914672463eed8f62)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:50.466216 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6368c311-b98b-4a4c-9a3e-659e2dca8c82 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:50.468052 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 212/410] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Sep 1 14:45:50 2022] GET /identity/v3/projects/b817b0f034a64d72914672463eed8f62 => generated 302 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:45:50.468932 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6368c311-b98b-4a4c-9a3e-659e2dca8c82 None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:50.469641 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ced432d-578b-4d7f-b6bb-85a53a0185a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:50.470392 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ced432d-578b-4d7f-b6bb-85a53a0185a1 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:50.470780 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ced432d-578b-4d7f-b6bb-85a53a0185a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:50.471146 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ced432d-578b-4d7f-b6bb-85a53a0185a1 None admin] PATH_INFO: `/v3/groups/a7d3f001853a464481317bfb0ee59f2f` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:50.472520 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ced432d-578b-4d7f-b6bb-85a53a0185a1 None admin] RBAC: Authorizing `identity:get_group(group_id=a7d3f001853a464481317bfb0ee59f2f)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:50.474784 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ced432d-578b-4d7f-b6bb-85a53a0185a1 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:50.479854 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 199/411] 10.222.0.33 () {56 vars in 1180 bytes} [Thu Sep 1 14:45:50 2022] GET /identity/v3/groups/a7d3f001853a464481317bfb0ee59f2f => generated 229 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:45:50.480726 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ced432d-578b-4d7f-b6bb-85a53a0185a1 None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:50.518918 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:50.599225 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ced432d-578b-4d7f-b6bb-85a53a0185a1 None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:50.603155 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6368c311-b98b-4a4c-9a3e-659e2dca8c82 None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:50.640858 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4d6916-a97f-422a-9599-26f905efd9a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:50.641768 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4d6916-a97f-422a-9599-26f905efd9a3 None admin] REQUEST_METHOD: `PUT` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:50.642050 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4d6916-a97f-422a-9599-26f905efd9a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:50.642348 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4d6916-a97f-422a-9599-26f905efd9a3 None admin] PATH_INFO: `/v3/projects/b817b0f034a64d72914672463eed8f62/groups/2bca6ad19b2849619ed210863be8b696/roles/9996407d588d4a00836c8c50e7f37b1a` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:50.644654 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4d6916-a97f-422a-9599-26f905efd9a3 None admin] RBAC: Authorizing `identity:create_grant(project_id=b817b0f034a64d72914672463eed8f62, group_id=2bca6ad19b2849619ed210863be8b696, role_id=9996407d588d4a00836c8c50e7f37b1a)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:50.646758 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4d6916-a97f-422a-9599-26f905efd9a3 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:50.648369 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd79a520-69d5-47c8-9f28-7aa9fe4ad3d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:50.649546 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd79a520-69d5-47c8-9f28-7aa9fe4ad3d9 None admin] REQUEST_METHOD: `PUT` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:50.650573 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd79a520-69d5-47c8-9f28-7aa9fe4ad3d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:50.654360 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd79a520-69d5-47c8-9f28-7aa9fe4ad3d9 None admin] PATH_INFO: `/v3/projects/b817b0f034a64d72914672463eed8f62/groups/2bca6ad19b2849619ed210863be8b696/roles/5d1feef0e13d4874928bd86258c4e3b2` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:50.657122 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd79a520-69d5-47c8-9f28-7aa9fe4ad3d9 None admin] RBAC: Authorizing `identity:create_grant(project_id=b817b0f034a64d72914672463eed8f62, group_id=2bca6ad19b2849619ed210863be8b696, role_id=5d1feef0e13d4874928bd86258c4e3b2)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:50.658955 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd79a520-69d5-47c8-9f28-7aa9fe4ad3d9 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:50.670395 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 200/412] 10.222.0.33 () {58 vars in 1442 bytes} [Thu Sep 1 14:45:50 2022] PUT /identity/v3/projects/b817b0f034a64d72914672463eed8f62/groups/2bca6ad19b2849619ed210863be8b696/roles/9996407d588d4a00836c8c50e7f37b1a => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:45:50.671640 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4d6916-a97f-422a-9599-26f905efd9a3 None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:50.688289 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 213/413] 10.222.0.33 () {58 vars in 1442 bytes} [Thu Sep 1 14:45:50 2022] PUT /identity/v3/projects/b817b0f034a64d72914672463eed8f62/groups/2bca6ad19b2849619ed210863be8b696/roles/5d1feef0e13d4874928bd86258c4e3b2 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:45:50.703523 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef95047-3aee-4e37-ad38-285db38c7b8b None None] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:50.704248 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef95047-3aee-4e37-ad38-285db38c7b8b None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:50.704765 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef95047-3aee-4e37-ad38-285db38c7b8b None None] PATH_INFO: `/` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:50.706826 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 214/414] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 1 14:45:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:45:50.718291 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c36b45-4178-44af-adf6-6592acb05f22 None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:50.718291 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c36b45-4178-44af-adf6-6592acb05f22 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:50.718291 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c36b45-4178-44af-adf6-6592acb05f22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:50.734129 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-90c36b45-4178-44af-adf6-6592acb05f22 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:45:50.735444 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-90c36b45-4178-44af-adf6-6592acb05f22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:45:50.763061 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:50.827685 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 215/415] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 1 14:45:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 14:45:50.883641 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:50.890379 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4d6916-a97f-422a-9599-26f905efd9a3 None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:50.925481 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8927693c-507e-419e-a20d-89f18b2480fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:50.926101 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8927693c-507e-419e-a20d-89f18b2480fa None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:50.926399 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8927693c-507e-419e-a20d-89f18b2480fa None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:50.926669 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8927693c-507e-419e-a20d-89f18b2480fa None admin] PATH_INFO: `/v3/projects/5cec9ba4a51d40d1be462a47013baf0a` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:50.927823 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8927693c-507e-419e-a20d-89f18b2480fa None admin] RBAC: Authorizing `identity:get_project(project_id=5cec9ba4a51d40d1be462a47013baf0a)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:50.929285 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8927693c-507e-419e-a20d-89f18b2480fa None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:50.931256 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 201/416] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Sep 1 14:45:50 2022] GET /identity/v3/projects/5cec9ba4a51d40d1be462a47013baf0a => generated 344 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:45:50.936962 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c36b45-4178-44af-adf6-6592acb05f22 None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:51.047041 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c36b45-4178-44af-adf6-6592acb05f22 None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:51.095984 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5ed730-e614-4050-8f37-d948f7e4a25f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:51.096932 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5ed730-e614-4050-8f37-d948f7e4a25f None admin] REQUEST_METHOD: `PUT` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:51.097390 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5ed730-e614-4050-8f37-d948f7e4a25f None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:51.097829 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5ed730-e614-4050-8f37-d948f7e4a25f None admin] PATH_INFO: `/v3/projects/5cec9ba4a51d40d1be462a47013baf0a/groups/a7d3f001853a464481317bfb0ee59f2f/roles/2a2adb769e55476b82e5014897e0a9c0` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:51.103498 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5ed730-e614-4050-8f37-d948f7e4a25f None admin] RBAC: Authorizing `identity:create_grant(project_id=5cec9ba4a51d40d1be462a47013baf0a, group_id=a7d3f001853a464481317bfb0ee59f2f, role_id=2a2adb769e55476b82e5014897e0a9c0)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:51.105123 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5ed730-e614-4050-8f37-d948f7e4a25f None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:51.123286 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 216/417] 10.222.0.33 () {58 vars in 1442 bytes} [Thu Sep 1 14:45:50 2022] PUT /identity/v3/projects/5cec9ba4a51d40d1be462a47013baf0a/groups/a7d3f001853a464481317bfb0ee59f2f/roles/2a2adb769e55476b82e5014897e0a9c0 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:45:51.145288 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8927693c-507e-419e-a20d-89f18b2480fa None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:51.259654 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8927693c-507e-419e-a20d-89f18b2480fa None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:51.297445 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7abf07-d38a-48a5-81d7-0c680e3804b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:51.298494 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7abf07-d38a-48a5-81d7-0c680e3804b2 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:51.299003 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7abf07-d38a-48a5-81d7-0c680e3804b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:51.299478 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7abf07-d38a-48a5-81d7-0c680e3804b2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:51.311368 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7abf07-d38a-48a5-81d7-0c680e3804b2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 14:45:51.317223 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application [None req-fe7abf07-d38a-48a5-81d7-0c680e3804b2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 14:45:51.317223 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:45:51.317223 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:45:51.317223 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:45:51.317223 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:45:51.317223 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:45:51.317223 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:45:51.317223 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:45:51.317223 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:45:51.317223 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:45:51.317223 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:45:51.317223 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:45:51.317223 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 14:45:51.317223 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 14:45:51.317223 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 14:45:51.317223 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 14:45:51.318790 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 14:45:51.318790 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 14:45:51.318790 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 14:45:51.318790 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 14:45:51.318790 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:45:51.318790 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:45:51.318790 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:45:51.318790 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:45:51.318790 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:45:51.318790 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:45:51.318790 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:45:51.318790 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application with Lock( Sep 01 14:45:51.318790 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:45:51.318790 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:45:51.318790 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:45:51.318790 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:45:51.319627 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:45:51.319627 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:45:51.319627 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:45:51.319627 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:45:51.319627 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 14:45:51.319627 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 14:45:51.319627 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 14:45:51.319627 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 14:45:51.319627 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 14:45:51.319627 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 14:45:51.319627 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 14:45:51.319627 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Sep 01 14:45:51.322691 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 202/418] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Sep 1 14:45:51 2022] GET /identity/v3/roles/member => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:45:51.331425 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5ed730-e614-4050-8f37-d948f7e4a25f None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:51.361830 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:51.454349 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5ed730-e614-4050-8f37-d948f7e4a25f None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:51.501655 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c578f4b-44c0-4cfe-9f86-43ad54eea9e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:51.502343 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c578f4b-44c0-4cfe-9f86-43ad54eea9e2 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:51.502625 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c578f4b-44c0-4cfe-9f86-43ad54eea9e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:51.502890 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c578f4b-44c0-4cfe-9f86-43ad54eea9e2 None admin] PATH_INFO: `/v3/roles` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:51.503556 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c578f4b-44c0-4cfe-9f86-43ad54eea9e2 None admin] RBAC: Adding query filter params (name=member) {{(pid=81342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:45:51.503854 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c578f4b-44c0-4cfe-9f86-43ad54eea9e2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:51.505212 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c578f4b-44c0-4cfe-9f86-43ad54eea9e2 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:51.514346 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 217/419] 10.222.0.33 () {56 vars in 1101 bytes} [Thu Sep 1 14:45:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:45:51.519893 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7abf07-d38a-48a5-81d7-0c680e3804b2 None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:51.563681 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:51.641252 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7abf07-d38a-48a5-81d7-0c680e3804b2 None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:51.649263 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efa7e25-586b-4a51-8f22-7629b6908040 None None] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:51.649263 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efa7e25-586b-4a51-8f22-7629b6908040 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:51.649829 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efa7e25-586b-4a51-8f22-7629b6908040 None None] PATH_INFO: `/` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:51.656089 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 218/420] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 1 14:45:51 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:45:51.664835 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc17eb9-16ea-470a-9ba6-194d345d6423 None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:51.665150 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc17eb9-16ea-470a-9ba6-194d345d6423 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:51.665375 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc17eb9-16ea-470a-9ba6-194d345d6423 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:51.679943 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15b0267-658c-44b3-8f3e-cb55d1212c85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:51.680551 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15b0267-658c-44b3-8f3e-cb55d1212c85 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:51.680831 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15b0267-658c-44b3-8f3e-cb55d1212c85 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:51.681100 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15b0267-658c-44b3-8f3e-cb55d1212c85 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:51.711219 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-0fc17eb9-16ea-470a-9ba6-194d345d6423 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:45:51.712747 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-0fc17eb9-16ea-470a-9ba6-194d345d6423 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:45:51.750658 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15b0267-658c-44b3-8f3e-cb55d1212c85 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:51.752152 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15b0267-658c-44b3-8f3e-cb55d1212c85 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:51.787662 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:51.857079 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 203/421] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Sep 1 14:45:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 14:45:51.862702 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15b0267-658c-44b3-8f3e-cb55d1212c85 None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:51.904562 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 219/422] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 1 14:45:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 14:45:51.966418 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15b0267-658c-44b3-8f3e-cb55d1212c85 None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:51.990604 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:52.001480 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16531974-aae7-4372-9627-1e4832805a89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:52.002585 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16531974-aae7-4372-9627-1e4832805a89 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:52.003239 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16531974-aae7-4372-9627-1e4832805a89 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:52.003512 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16531974-aae7-4372-9627-1e4832805a89 None admin] PATH_INFO: `/v3/projects/1f696f55becc4e879b6d1bac9d93d055` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:52.004662 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16531974-aae7-4372-9627-1e4832805a89 None admin] RBAC: Authorizing `identity:get_project(project_id=1f696f55becc4e879b6d1bac9d93d055)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:52.006172 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16531974-aae7-4372-9627-1e4832805a89 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:52.008190 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 204/423] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Sep 1 14:45:51 2022] GET /identity/v3/projects/1f696f55becc4e879b6d1bac9d93d055 => generated 298 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 14:45:52.013723 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc17eb9-16ea-470a-9ba6-194d345d6423 None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:52.030587 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:52.149524 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc17eb9-16ea-470a-9ba6-194d345d6423 None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:52.197597 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a83dd1f-9ea9-4d15-bf64-e4c990528b22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:52.198155 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a83dd1f-9ea9-4d15-bf64-e4c990528b22 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:52.198461 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a83dd1f-9ea9-4d15-bf64-e4c990528b22 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:52.198736 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a83dd1f-9ea9-4d15-bf64-e4c990528b22 None admin] PATH_INFO: `/v3/groups/2bca6ad19b2849619ed210863be8b696` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:52.199913 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a83dd1f-9ea9-4d15-bf64-e4c990528b22 None admin] RBAC: Authorizing `identity:get_group(group_id=2bca6ad19b2849619ed210863be8b696)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:52.201361 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a83dd1f-9ea9-4d15-bf64-e4c990528b22 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:52.203258 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 220/424] 10.222.0.33 () {56 vars in 1180 bytes} [Thu Sep 1 14:45:52 2022] GET /identity/v3/groups/2bca6ad19b2849619ed210863be8b696 => generated 226 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:45:52.209121 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16531974-aae7-4372-9627-1e4832805a89 None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:52.239174 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a83dd1f-9ea9-4d15-bf64-e4c990528b22 None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:52.334625 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:52.358621 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a83dd1f-9ea9-4d15-bf64-e4c990528b22 None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:52.403730 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16531974-aae7-4372-9627-1e4832805a89 None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:52.419267 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53d125e-5819-4abd-a826-e7ab9df651ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:52.419896 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53d125e-5819-4abd-a826-e7ab9df651ae None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:52.420174 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53d125e-5819-4abd-a826-e7ab9df651ae None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:52.420444 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53d125e-5819-4abd-a826-e7ab9df651ae None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:52.439913 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53d125e-5819-4abd-a826-e7ab9df651ae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 14:45:52.446844 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application [None req-c53d125e-5819-4abd-a826-e7ab9df651ae None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 14:45:52.446844 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:45:52.446844 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:45:52.446844 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:45:52.446844 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:45:52.446844 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:45:52.446844 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:45:52.446844 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:45:52.446844 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:45:52.446844 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:45:52.446844 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:45:52.446844 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:45:52.446844 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 14:45:52.446844 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 14:45:52.446844 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 14:45:52.446844 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 14:45:52.447892 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 14:45:52.447892 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 14:45:52.447892 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 14:45:52.447892 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 14:45:52.447892 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:45:52.447892 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:45:52.447892 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:45:52.447892 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:45:52.447892 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:45:52.447892 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:45:52.447892 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:45:52.447892 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application with Lock( Sep 01 14:45:52.447892 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:45:52.447892 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:45:52.447892 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:45:52.447892 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:45:52.449274 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:45:52.449274 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:45:52.449274 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:45:52.449274 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:45:52.449274 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 14:45:52.449274 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 14:45:52.449274 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 14:45:52.449274 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 14:45:52.449274 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 14:45:52.449274 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 14:45:52.449274 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 14:45:52.449274 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Sep 01 14:45:52.449274 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 221/425] 10.222.0.33 () {56 vars in 1114 bytes} [Thu Sep 1 14:45:52 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:45:52.454207 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53d125e-5819-4abd-a826-e7ab9df651ae None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:52.473587 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1d993a-9360-4dcf-a25b-a36543e4e7c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:52.474785 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1d993a-9360-4dcf-a25b-a36543e4e7c0 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:52.475074 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1d993a-9360-4dcf-a25b-a36543e4e7c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:52.475342 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1d993a-9360-4dcf-a25b-a36543e4e7c0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:52.476075 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1d993a-9360-4dcf-a25b-a36543e4e7c0 None admin] RBAC: Adding query filter params (group.id=2bca6ad19b2849619ed210863be8b696, role.id=5d1feef0e13d4874928bd86258c4e3b2, scope.project.id=1f696f55becc4e879b6d1bac9d93d055) {{(pid=81341) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:45:52.476380 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1d993a-9360-4dcf-a25b-a36543e4e7c0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:52.477837 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1d993a-9360-4dcf-a25b-a36543e4e7c0 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:52.489236 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 205/426] 10.222.0.33 () {56 vars in 1376 bytes} [Thu Sep 1 14:45:52 2022] GET /identity/v3/role_assignments?group.id=2bca6ad19b2849619ed210863be8b696&scope.project.id=1f696f55becc4e879b6d1bac9d93d055&role.id=5d1feef0e13d4874928bd86258c4e3b2 => generated 611 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:45:52.625034 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53d125e-5819-4abd-a826-e7ab9df651ae None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:52.680936 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97bf023-6a03-4a05-9957-ac698cfa98b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:52.681595 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97bf023-6a03-4a05-9957-ac698cfa98b6 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:52.681831 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97bf023-6a03-4a05-9957-ac698cfa98b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:52.682103 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97bf023-6a03-4a05-9957-ac698cfa98b6 None admin] PATH_INFO: `/v3/roles` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:52.682817 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97bf023-6a03-4a05-9957-ac698cfa98b6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:45:52.683122 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97bf023-6a03-4a05-9957-ac698cfa98b6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:52.684544 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97bf023-6a03-4a05-9957-ac698cfa98b6 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:52.694984 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:52.699942 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 222/427] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Sep 1 14:45:52 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:45:52.705613 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1d993a-9360-4dcf-a25b-a36543e4e7c0 None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:52.769339 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:52.820625 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1d993a-9360-4dcf-a25b-a36543e4e7c0 None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:52.860714 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b795f7d-e0c6-4006-912e-9d5ef5ada976 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:52.861319 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b795f7d-e0c6-4006-912e-9d5ef5ada976 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:52.861591 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b795f7d-e0c6-4006-912e-9d5ef5ada976 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:52.861857 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b795f7d-e0c6-4006-912e-9d5ef5ada976 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:52.935677 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b795f7d-e0c6-4006-912e-9d5ef5ada976 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:52.937161 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b795f7d-e0c6-4006-912e-9d5ef5ada976 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:52.995845 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:53.046276 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 206/428] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Sep 1 14:45:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 14:45:53.054308 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97bf023-6a03-4a05-9957-ac698cfa98b6 None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:53.146683 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f64e4b-102f-4ca1-b154-6cf656d56116 None None] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:53.147298 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f64e4b-102f-4ca1-b154-6cf656d56116 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:53.147892 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f64e4b-102f-4ca1-b154-6cf656d56116 None None] PATH_INFO: `/` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:53.149814 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 207/429] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 1 14:45:53 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:45:53.154904 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db2969b-5a33-4b4f-a953-ec1ebbd2b377 None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:53.155174 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db2969b-5a33-4b4f-a953-ec1ebbd2b377 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:53.155446 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db2969b-5a33-4b4f-a953-ec1ebbd2b377 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:53.181164 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-8db2969b-5a33-4b4f-a953-ec1ebbd2b377 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:45:53.182401 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-8db2969b-5a33-4b4f-a953-ec1ebbd2b377 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:45:53.191546 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97bf023-6a03-4a05-9957-ac698cfa98b6 None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:53.243335 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46234670-f532-44ee-a62d-a40c1c27cc3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:53.243978 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46234670-f532-44ee-a62d-a40c1c27cc3f None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:53.244262 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46234670-f532-44ee-a62d-a40c1c27cc3f None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:53.244534 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46234670-f532-44ee-a62d-a40c1c27cc3f None admin] PATH_INFO: `/v3/projects/1f696f55becc4e879b6d1bac9d93d055` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:53.246171 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46234670-f532-44ee-a62d-a40c1c27cc3f None admin] RBAC: Authorizing `identity:get_project(project_id=1f696f55becc4e879b6d1bac9d93d055)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:53.247702 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46234670-f532-44ee-a62d-a40c1c27cc3f None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:53.249708 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 223/430] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Sep 1 14:45:53 2022] GET /identity/v3/projects/1f696f55becc4e879b6d1bac9d93d055 => generated 298 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:45:53.250615 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbea75a-6ca7-47e2-ab6a-6e46d985cb8e None None] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:53.250900 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbea75a-6ca7-47e2-ab6a-6e46d985cb8e None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:53.251689 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbea75a-6ca7-47e2-ab6a-6e46d985cb8e None None] PATH_INFO: `/` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:53.252356 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 224/431] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 1 14:45:53 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:45:53.259752 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbea75a-6ca7-47e2-ab6a-6e46d985cb8e None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:53.268047 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 208/432] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 1 14:45:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 14:45:53.269583 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc844c87-2837-4cca-a072-129afdda0824 None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:53.270036 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc844c87-2837-4cca-a072-129afdda0824 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:53.270560 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc844c87-2837-4cca-a072-129afdda0824 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:53.289490 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-cc844c87-2837-4cca-a072-129afdda0824 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:45:53.290743 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-cc844c87-2837-4cca-a072-129afdda0824 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:45:53.345127 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:53.370914 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 209/433] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 1 14:45:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 14:45:53.392768 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbea75a-6ca7-47e2-ab6a-6e46d985cb8e None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:53.428070 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf12818-8637-435c-b349-e04890e97494 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:53.428982 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf12818-8637-435c-b349-e04890e97494 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:53.429406 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf12818-8637-435c-b349-e04890e97494 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:53.429900 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf12818-8637-435c-b349-e04890e97494 None admin] PATH_INFO: `/v3/groups/2bca6ad19b2849619ed210863be8b696` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:53.431255 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf12818-8637-435c-b349-e04890e97494 None admin] RBAC: Authorizing `identity:get_group(group_id=2bca6ad19b2849619ed210863be8b696)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:53.432875 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf12818-8637-435c-b349-e04890e97494 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:53.434940 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 225/434] 10.222.0.33 () {56 vars in 1180 bytes} [Thu Sep 1 14:45:53 2022] GET /identity/v3/groups/2bca6ad19b2849619ed210863be8b696 => generated 226 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:45:53.440714 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc844c87-2837-4cca-a072-129afdda0824 None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:53.468120 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:53.550177 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc844c87-2837-4cca-a072-129afdda0824 None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:53.589701 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959d9011-d9d9-4a95-a616-390e9bb38cf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:53.590425 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959d9011-d9d9-4a95-a616-390e9bb38cf4 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:53.590700 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959d9011-d9d9-4a95-a616-390e9bb38cf4 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:53.590964 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959d9011-d9d9-4a95-a616-390e9bb38cf4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:53.591688 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959d9011-d9d9-4a95-a616-390e9bb38cf4 None admin] RBAC: Adding query filter params (group.id=2bca6ad19b2849619ed210863be8b696, role.id=9996407d588d4a00836c8c50e7f37b1a, scope.project.id=1f696f55becc4e879b6d1bac9d93d055) {{(pid=81341) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:45:53.591792 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61059f58-fa0e-4b17-9b3d-3e4786b9f0bf None None] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:53.591990 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959d9011-d9d9-4a95-a616-390e9bb38cf4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:53.592085 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61059f58-fa0e-4b17-9b3d-3e4786b9f0bf None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:53.592345 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61059f58-fa0e-4b17-9b3d-3e4786b9f0bf None None] PATH_INFO: `/` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:53.593429 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959d9011-d9d9-4a95-a616-390e9bb38cf4 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:53.593918 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 226/435] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 1 14:45:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:45:53.599604 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7393938f-ebac-4cd5-b74a-4910d3de07f3 None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:53.599881 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7393938f-ebac-4cd5-b74a-4910d3de07f3 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:53.600148 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7393938f-ebac-4cd5-b74a-4910d3de07f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:53.604075 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 210/436] 10.222.0.33 () {56 vars in 1376 bytes} [Thu Sep 1 14:45:53 2022] GET /identity/v3/role_assignments?group.id=2bca6ad19b2849619ed210863be8b696&scope.project.id=1f696f55becc4e879b6d1bac9d93d055&role.id=9996407d588d4a00836c8c50e7f37b1a => generated 611 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:45:53.619904 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-7393938f-ebac-4cd5-b74a-4910d3de07f3 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:45:53.621126 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-7393938f-ebac-4cd5-b74a-4910d3de07f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:45:53.623197 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959d9011-d9d9-4a95-a616-390e9bb38cf4 None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:53.702754 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 227/437] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 1 14:45:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 14:45:53.703648 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7393938f-ebac-4cd5-b74a-4910d3de07f3 None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:53.734412 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:53.762829 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959d9011-d9d9-4a95-a616-390e9bb38cf4 None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:53.792330 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:53.810070 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afaffc6-2f39-46aa-972c-5e3730561fdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:53.811068 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afaffc6-2f39-46aa-972c-5e3730561fdd None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:53.811488 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afaffc6-2f39-46aa-972c-5e3730561fdd None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:53.811916 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afaffc6-2f39-46aa-972c-5e3730561fdd None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:53.829381 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3afaffc6-2f39-46aa-972c-5e3730561fdd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 14:45:53.835703 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application [None req-3afaffc6-2f39-46aa-972c-5e3730561fdd None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 14:45:53.835703 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:45:53.835703 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:45:53.835703 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:45:53.835703 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:45:53.835703 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:45:53.835703 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:45:53.835703 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:45:53.835703 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:45:53.835703 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:45:53.835703 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:45:53.835703 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:45:53.835703 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 14:45:53.835703 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 14:45:53.835703 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 14:45:53.835703 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 14:45:53.836802 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 14:45:53.836802 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 14:45:53.836802 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 14:45:53.836802 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 14:45:53.836802 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:45:53.836802 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:45:53.836802 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:45:53.836802 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:45:53.836802 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:45:53.836802 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:45:53.836802 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:45:53.836802 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application with Lock( Sep 01 14:45:53.836802 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:45:53.836802 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:45:53.836802 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:45:53.836802 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:45:53.837721 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:45:53.837721 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:45:53.837721 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:45:53.837721 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:45:53.837721 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 14:45:53.837721 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 14:45:53.837721 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 14:45:53.837721 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 14:45:53.837721 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 14:45:53.837721 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 14:45:53.837721 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 14:45:53.837721 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Sep 01 14:45:53.837721 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 211/438] 10.222.0.33 () {56 vars in 1114 bytes} [Thu Sep 1 14:45:53 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 14:45:53.839916 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7393938f-ebac-4cd5-b74a-4910d3de07f3 None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:53.843387 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afaffc6-2f39-46aa-972c-5e3730561fdd None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:53.887805 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9574dd4c-5bab-407f-a306-c775fbdb2007 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:53.888818 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9574dd4c-5bab-407f-a306-c775fbdb2007 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:53.889299 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9574dd4c-5bab-407f-a306-c775fbdb2007 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:53.889765 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9574dd4c-5bab-407f-a306-c775fbdb2007 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:53.902547 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9574dd4c-5bab-407f-a306-c775fbdb2007 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 14:45:53.907754 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application [None req-9574dd4c-5bab-407f-a306-c775fbdb2007 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 14:45:53.907754 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:45:53.907754 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:45:53.907754 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:45:53.907754 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:45:53.907754 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:45:53.907754 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:45:53.907754 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:45:53.907754 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:45:53.907754 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:45:53.907754 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:45:53.907754 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:45:53.907754 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 14:45:53.907754 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 14:45:53.907754 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 14:45:53.907754 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 14:45:53.908661 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 14:45:53.908661 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 14:45:53.908661 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 14:45:53.908661 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 14:45:53.908661 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:45:53.908661 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:45:53.908661 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:45:53.908661 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:45:53.908661 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:45:53.908661 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:45:53.908661 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:45:53.908661 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application with Lock( Sep 01 14:45:53.908661 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:45:53.908661 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:45:53.908661 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:45:53.908661 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:45:53.909504 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:45:53.909504 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:45:53.909504 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:45:53.909504 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:45:53.909504 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 14:45:53.909504 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 14:45:53.909504 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 14:45:53.909504 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 14:45:53.909504 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 14:45:53.909504 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 14:45:53.909504 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 14:45:53.909504 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Sep 01 14:45:53.909504 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 228/439] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Sep 1 14:45:53 2022] GET /identity/v3/roles/member => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:45:53.916958 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9574dd4c-5bab-407f-a306-c775fbdb2007 None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:53.976090 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afaffc6-2f39-46aa-972c-5e3730561fdd None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:54.015445 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557de939-b602-4537-8bb9-4ce3b9021007 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:54.016088 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557de939-b602-4537-8bb9-4ce3b9021007 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:54.016309 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557de939-b602-4537-8bb9-4ce3b9021007 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:54.016582 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557de939-b602-4537-8bb9-4ce3b9021007 None admin] PATH_INFO: `/v3/roles` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:54.017689 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557de939-b602-4537-8bb9-4ce3b9021007 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81341) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:45:54.017689 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557de939-b602-4537-8bb9-4ce3b9021007 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:54.019324 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557de939-b602-4537-8bb9-4ce3b9021007 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:54.029455 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 212/440] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Sep 1 14:45:53 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 14:45:54.029455 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557de939-b602-4537-8bb9-4ce3b9021007 None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:54.039017 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:54.046286 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9574dd4c-5bab-407f-a306-c775fbdb2007 None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:54.071407 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:54.082460 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b63d4f-114d-4a6c-b3b3-9478e783aba9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:54.083069 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b63d4f-114d-4a6c-b3b3-9478e783aba9 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:54.083864 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b63d4f-114d-4a6c-b3b3-9478e783aba9 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:54.084139 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b63d4f-114d-4a6c-b3b3-9478e783aba9 None admin] PATH_INFO: `/v3/roles` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:54.084806 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b63d4f-114d-4a6c-b3b3-9478e783aba9 None admin] RBAC: Adding query filter params (name=member) {{(pid=81342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:45:54.085106 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b63d4f-114d-4a6c-b3b3-9478e783aba9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:54.086523 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b63d4f-114d-4a6c-b3b3-9478e783aba9 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:54.094699 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 229/441] 10.222.0.33 () {56 vars in 1101 bytes} [Thu Sep 1 14:45:53 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:45:54.095463 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b63d4f-114d-4a6c-b3b3-9478e783aba9 None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:54.144784 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557de939-b602-4537-8bb9-4ce3b9021007 None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:54.178072 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b700f7-b280-4755-8998-6709b628876a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:54.178719 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b700f7-b280-4755-8998-6709b628876a None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:54.179000 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b700f7-b280-4755-8998-6709b628876a None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:54.179270 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b700f7-b280-4755-8998-6709b628876a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:54.188982 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b700f7-b280-4755-8998-6709b628876a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 14:45:54.191042 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b63d4f-114d-4a6c-b3b3-9478e783aba9 None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:54.194161 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application [None req-91b700f7-b280-4755-8998-6709b628876a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 14:45:54.194161 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:45:54.194161 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:45:54.194161 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:45:54.194161 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:45:54.194161 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:45:54.194161 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:45:54.194161 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:45:54.194161 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:45:54.194161 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:45:54.194161 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:45:54.194161 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:45:54.194161 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 14:45:54.194161 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 14:45:54.194161 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 14:45:54.194161 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 14:45:54.195345 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 14:45:54.195345 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 14:45:54.195345 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 14:45:54.195345 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 14:45:54.195345 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:45:54.195345 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:45:54.195345 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:45:54.195345 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:45:54.195345 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:45:54.195345 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:45:54.195345 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:45:54.195345 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application with Lock( Sep 01 14:45:54.195345 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:45:54.195345 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:45:54.195345 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:45:54.195345 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:45:54.196439 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:45:54.196439 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:45:54.196439 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:45:54.196439 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:45:54.196439 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 14:45:54.196439 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 14:45:54.196439 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 14:45:54.196439 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 14:45:54.196439 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 14:45:54.196439 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 14:45:54.196439 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 14:45:54.196439 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Sep 01 14:45:54.196439 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 213/442] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Sep 1 14:45:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:45:54.196439 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b700f7-b280-4755-8998-6709b628876a None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:54.225238 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889ec8d4-15e3-4150-a4cf-2a9a84bab340 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:54.225843 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889ec8d4-15e3-4150-a4cf-2a9a84bab340 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:54.226122 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889ec8d4-15e3-4150-a4cf-2a9a84bab340 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:54.226433 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889ec8d4-15e3-4150-a4cf-2a9a84bab340 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:54.256323 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:54.280027 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:54.293943 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889ec8d4-15e3-4150-a4cf-2a9a84bab340 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:54.295175 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b700f7-b280-4755-8998-6709b628876a None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:54.295439 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889ec8d4-15e3-4150-a4cf-2a9a84bab340 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:54.328368 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde4f399-10e6-43e7-aea7-3f14a8e59aac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:54.328883 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde4f399-10e6-43e7-aea7-3f14a8e59aac None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:54.329163 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde4f399-10e6-43e7-aea7-3f14a8e59aac None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:54.329436 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde4f399-10e6-43e7-aea7-3f14a8e59aac None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:54.390910 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 230/443] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Sep 1 14:45:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 14:45:54.391734 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889ec8d4-15e3-4150-a4cf-2a9a84bab340 None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:54.395270 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde4f399-10e6-43e7-aea7-3f14a8e59aac None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:54.397689 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde4f399-10e6-43e7-aea7-3f14a8e59aac None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:54.455049 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:54.476614 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:54.491738 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889ec8d4-15e3-4150-a4cf-2a9a84bab340 None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:54.493293 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 214/444] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Sep 1 14:45:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 14:45:54.494105 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde4f399-10e6-43e7-aea7-3f14a8e59aac None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:54.531556 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f483d6-4a12-4dbc-b6b6-6f7a6f0a496b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:54.532156 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f483d6-4a12-4dbc-b6b6-6f7a6f0a496b None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:54.532435 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f483d6-4a12-4dbc-b6b6-6f7a6f0a496b None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:54.532702 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f483d6-4a12-4dbc-b6b6-6f7a6f0a496b None admin] PATH_INFO: `/v3/roles` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:54.533358 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f483d6-4a12-4dbc-b6b6-6f7a6f0a496b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:45:54.533653 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f483d6-4a12-4dbc-b6b6-6f7a6f0a496b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:54.535042 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f483d6-4a12-4dbc-b6b6-6f7a6f0a496b None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:54.542816 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 231/445] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Sep 1 14:45:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:45:54.543578 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f483d6-4a12-4dbc-b6b6-6f7a6f0a496b None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:54.590994 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde4f399-10e6-43e7-aea7-3f14a8e59aac None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:54.623904 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4b25da-78bc-4354-b0e0-4579b394726e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:54.624523 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4b25da-78bc-4354-b0e0-4579b394726e None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:54.624802 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4b25da-78bc-4354-b0e0-4579b394726e None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:54.625077 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4b25da-78bc-4354-b0e0-4579b394726e None admin] PATH_INFO: `/v3/projects/b817b0f034a64d72914672463eed8f62` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:54.626215 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4b25da-78bc-4354-b0e0-4579b394726e None admin] RBAC: Authorizing `identity:get_project(project_id=b817b0f034a64d72914672463eed8f62)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:54.627725 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4b25da-78bc-4354-b0e0-4579b394726e None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:54.629317 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 215/446] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Sep 1 14:45:54 2022] GET /identity/v3/projects/b817b0f034a64d72914672463eed8f62 => generated 302 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:45:54.630129 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4b25da-78bc-4354-b0e0-4579b394726e None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:54.644128 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f483d6-4a12-4dbc-b6b6-6f7a6f0a496b None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:54.667225 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:54.680598 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bd5e94-d7e8-4bb9-bb92-de4edded49dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:54.681216 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bd5e94-d7e8-4bb9-bb92-de4edded49dc None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:54.681546 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bd5e94-d7e8-4bb9-bb92-de4edded49dc None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:54.681819 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bd5e94-d7e8-4bb9-bb92-de4edded49dc None admin] PATH_INFO: `/v3/projects/b817b0f034a64d72914672463eed8f62` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:54.682959 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bd5e94-d7e8-4bb9-bb92-de4edded49dc None admin] RBAC: Authorizing `identity:get_project(project_id=b817b0f034a64d72914672463eed8f62)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:54.684404 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bd5e94-d7e8-4bb9-bb92-de4edded49dc None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:54.686003 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 232/447] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Sep 1 14:45:54 2022] GET /identity/v3/projects/b817b0f034a64d72914672463eed8f62 => generated 302 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:45:54.686003 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bd5e94-d7e8-4bb9-bb92-de4edded49dc None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:54.704736 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:54.741680 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4b25da-78bc-4354-b0e0-4579b394726e None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:54.775444 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1b795d-faea-4c11-a766-a1e1222ddd75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:54.776128 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1b795d-faea-4c11-a766-a1e1222ddd75 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:54.776409 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1b795d-faea-4c11-a766-a1e1222ddd75 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:54.776680 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1b795d-faea-4c11-a766-a1e1222ddd75 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:54.787363 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bd5e94-d7e8-4bb9-bb92-de4edded49dc None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:54.821567 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f400e360-c50f-4658-8b8d-24e4b5abc03c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:54.822180 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f400e360-c50f-4658-8b8d-24e4b5abc03c None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:54.822485 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f400e360-c50f-4658-8b8d-24e4b5abc03c None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:54.822754 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f400e360-c50f-4658-8b8d-24e4b5abc03c None admin] PATH_INFO: `/v3/groups/2bca6ad19b2849619ed210863be8b696` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:54.823876 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f400e360-c50f-4658-8b8d-24e4b5abc03c None admin] RBAC: Authorizing `identity:get_group(group_id=2bca6ad19b2849619ed210863be8b696)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:54.825382 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f400e360-c50f-4658-8b8d-24e4b5abc03c None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:54.827165 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 233/448] 10.222.0.33 () {56 vars in 1180 bytes} [Thu Sep 1 14:45:54 2022] GET /identity/v3/groups/2bca6ad19b2849619ed210863be8b696 => generated 226 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:45:54.827763 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f400e360-c50f-4658-8b8d-24e4b5abc03c None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:54.846957 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1b795d-faea-4c11-a766-a1e1222ddd75 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:54.848362 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1b795d-faea-4c11-a766-a1e1222ddd75 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:54.876882 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:54.911918 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:54.928629 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f400e360-c50f-4658-8b8d-24e4b5abc03c None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:54.947692 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 216/449] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Sep 1 14:45:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 14:45:54.948518 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1b795d-faea-4c11-a766-a1e1222ddd75 None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:54.962153 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e26e93c-6b67-4531-8f83-9b72ce60cf52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:54.962829 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e26e93c-6b67-4531-8f83-9b72ce60cf52 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:54.963104 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e26e93c-6b67-4531-8f83-9b72ce60cf52 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:54.963370 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e26e93c-6b67-4531-8f83-9b72ce60cf52 None admin] PATH_INFO: `/v3/groups/2bca6ad19b2849619ed210863be8b696` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:54.964454 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e26e93c-6b67-4531-8f83-9b72ce60cf52 None admin] RBAC: Authorizing `identity:get_group(group_id=2bca6ad19b2849619ed210863be8b696)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:54.965866 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e26e93c-6b67-4531-8f83-9b72ce60cf52 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:54.967571 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 234/450] 10.222.0.33 () {56 vars in 1180 bytes} [Thu Sep 1 14:45:54 2022] GET /identity/v3/groups/2bca6ad19b2849619ed210863be8b696 => generated 226 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:45:54.968236 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e26e93c-6b67-4531-8f83-9b72ce60cf52 None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:55.048213 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1b795d-faea-4c11-a766-a1e1222ddd75 None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:55.064927 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e26e93c-6b67-4531-8f83-9b72ce60cf52 None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:55.077023 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:55.083161 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebded4e-88e4-4d68-b984-5851be7c2c2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:55.083759 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebded4e-88e4-4d68-b984-5851be7c2c2d None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:55.084038 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebded4e-88e4-4d68-b984-5851be7c2c2d None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:55.084306 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebded4e-88e4-4d68-b984-5851be7c2c2d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:55.085029 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebded4e-88e4-4d68-b984-5851be7c2c2d None admin] RBAC: Adding query filter params (group.id=2bca6ad19b2849619ed210863be8b696, role.id=9996407d588d4a00836c8c50e7f37b1a, scope.project.id=b817b0f034a64d72914672463eed8f62) {{(pid=81341) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:45:55.085337 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebded4e-88e4-4d68-b984-5851be7c2c2d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:55.086813 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebded4e-88e4-4d68-b984-5851be7c2c2d None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:55.092391 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 217/451] 10.222.0.33 () {56 vars in 1376 bytes} [Thu Sep 1 14:45:54 2022] GET /identity/v3/role_assignments?group.id=2bca6ad19b2849619ed210863be8b696&scope.project.id=b817b0f034a64d72914672463eed8f62&role.id=9996407d588d4a00836c8c50e7f37b1a => generated 611 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:45:55.093158 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebded4e-88e4-4d68-b984-5851be7c2c2d None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:55.098298 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e48cc2e-b3be-40f3-bf91-3ce2d53a68af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:55.099398 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e48cc2e-b3be-40f3-bf91-3ce2d53a68af None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:55.099691 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e48cc2e-b3be-40f3-bf91-3ce2d53a68af None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:55.099962 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e48cc2e-b3be-40f3-bf91-3ce2d53a68af None admin] PATH_INFO: `/v3/projects/5cec9ba4a51d40d1be462a47013baf0a` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:55.101154 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e48cc2e-b3be-40f3-bf91-3ce2d53a68af None admin] RBAC: Authorizing `identity:get_project(project_id=5cec9ba4a51d40d1be462a47013baf0a)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:55.102663 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e48cc2e-b3be-40f3-bf91-3ce2d53a68af None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:55.104374 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 235/452] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Sep 1 14:45:54 2022] GET /identity/v3/projects/5cec9ba4a51d40d1be462a47013baf0a => generated 344 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:45:55.110265 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e48cc2e-b3be-40f3-bf91-3ce2d53a68af None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:55.134532 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:55.218028 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebded4e-88e4-4d68-b984-5851be7c2c2d None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:55.221992 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e48cc2e-b3be-40f3-bf91-3ce2d53a68af None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:55.259655 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02a9764-adad-4990-9634-2b6376663498 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:55.260268 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02a9764-adad-4990-9634-2b6376663498 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:55.260536 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02a9764-adad-4990-9634-2b6376663498 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:55.260805 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02a9764-adad-4990-9634-2b6376663498 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:55.261526 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02a9764-adad-4990-9634-2b6376663498 None admin] RBAC: Adding query filter params (group.id=2bca6ad19b2849619ed210863be8b696, role.id=5d1feef0e13d4874928bd86258c4e3b2, scope.project.id=b817b0f034a64d72914672463eed8f62) {{(pid=81341) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:45:55.261828 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02a9764-adad-4990-9634-2b6376663498 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:55.263193 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7920bf55-a81c-4afe-9769-dc58f80ef820 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:55.263361 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02a9764-adad-4990-9634-2b6376663498 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:55.263824 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7920bf55-a81c-4afe-9769-dc58f80ef820 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:55.264106 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7920bf55-a81c-4afe-9769-dc58f80ef820 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:55.264377 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7920bf55-a81c-4afe-9769-dc58f80ef820 None admin] PATH_INFO: `/v3/groups/a7d3f001853a464481317bfb0ee59f2f` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:55.265639 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7920bf55-a81c-4afe-9769-dc58f80ef820 None admin] RBAC: Authorizing `identity:get_group(group_id=a7d3f001853a464481317bfb0ee59f2f)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:55.267664 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7920bf55-a81c-4afe-9769-dc58f80ef820 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:55.269556 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 236/453] 10.222.0.33 () {56 vars in 1180 bytes} [Thu Sep 1 14:45:55 2022] GET /identity/v3/groups/a7d3f001853a464481317bfb0ee59f2f => generated 229 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 14:45:55.269783 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 218/454] 10.222.0.33 () {56 vars in 1376 bytes} [Thu Sep 1 14:45:55 2022] GET /identity/v3/role_assignments?group.id=2bca6ad19b2849619ed210863be8b696&scope.project.id=b817b0f034a64d72914672463eed8f62&role.id=5d1feef0e13d4874928bd86258c4e3b2 => generated 611 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:45:55.278080 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7920bf55-a81c-4afe-9769-dc58f80ef820 None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:55.372664 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:55.385575 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7920bf55-a81c-4afe-9769-dc58f80ef820 None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:55.420486 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19edf31-5e57-42da-a26a-0697079caa42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:55.421093 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19edf31-5e57-42da-a26a-0697079caa42 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:55.421370 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19edf31-5e57-42da-a26a-0697079caa42 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:55.421638 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19edf31-5e57-42da-a26a-0697079caa42 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:55.422444 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19edf31-5e57-42da-a26a-0697079caa42 None admin] RBAC: Adding query filter params (group.id=a7d3f001853a464481317bfb0ee59f2f, role.id=2a2adb769e55476b82e5014897e0a9c0, scope.project.id=5cec9ba4a51d40d1be462a47013baf0a) {{(pid=81342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:45:55.422751 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19edf31-5e57-42da-a26a-0697079caa42 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:55.424198 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19edf31-5e57-42da-a26a-0697079caa42 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:55.430678 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 237/455] 10.222.0.33 () {56 vars in 1376 bytes} [Thu Sep 1 14:45:55 2022] GET /identity/v3/role_assignments?group.id=a7d3f001853a464481317bfb0ee59f2f&scope.project.id=5cec9ba4a51d40d1be462a47013baf0a&role.id=2a2adb769e55476b82e5014897e0a9c0 => generated 611 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:45:57.796549 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a99b4b6-27f1-48a6-aa20-c1a27c166245 None None] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:57.797073 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a99b4b6-27f1-48a6-aa20-c1a27c166245 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:57.797073 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a99b4b6-27f1-48a6-aa20-c1a27c166245 None None] PATH_INFO: `/` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:57.798732 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 219/456] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 1 14:45:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:45:57.811762 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a231fe01-4bd4-42d0-b231-df98a6cd72dd None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:57.812550 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a231fe01-4bd4-42d0-b231-df98a6cd72dd None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:57.813067 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a231fe01-4bd4-42d0-b231-df98a6cd72dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:57.842476 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-a231fe01-4bd4-42d0-b231-df98a6cd72dd None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:45:57.843900 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-a231fe01-4bd4-42d0-b231-df98a6cd72dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:45:57.934433 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 238/457] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 1 14:45:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 14:45:58.037257 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d38ea4-76b0-4ba7-b87c-d8d6f966dcee None None] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:58.037536 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d38ea4-76b0-4ba7-b87c-d8d6f966dcee None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:58.037970 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d38ea4-76b0-4ba7-b87c-d8d6f966dcee None None] PATH_INFO: `/` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:58.039497 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 220/458] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 1 14:45:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:45:58.044829 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f034d35f-653d-47cd-82a2-df63aaa1ace0 None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:58.045218 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f034d35f-653d-47cd-82a2-df63aaa1ace0 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:58.045494 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f034d35f-653d-47cd-82a2-df63aaa1ace0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:58.075290 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-f034d35f-653d-47cd-82a2-df63aaa1ace0 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:45:58.076512 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-f034d35f-653d-47cd-82a2-df63aaa1ace0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:45:58.167873 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 239/459] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 1 14:45:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 14:45:58.358431 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d38ea4-76b0-4ba7-b87c-d8d6f966dcee None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:58.394899 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b6ba4e-5495-496f-a425-6f6a369e625e None None] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:58.395434 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b6ba4e-5495-496f-a425-6f6a369e625e None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:58.395882 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b6ba4e-5495-496f-a425-6f6a369e625e None None] PATH_INFO: `/` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:58.397602 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 240/460] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 1 14:45:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:45:58.402376 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0582264-4be2-48db-82df-7dfc32479160 None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:58.402949 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0582264-4be2-48db-82df-7dfc32479160 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:58.403439 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0582264-4be2-48db-82df-7dfc32479160 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:58.432630 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-c0582264-4be2-48db-82df-7dfc32479160 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:45:58.433845 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-c0582264-4be2-48db-82df-7dfc32479160 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:45:58.449423 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:58.457509 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:58.534322 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d38ea4-76b0-4ba7-b87c-d8d6f966dcee None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:58.551994 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 241/461] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 1 14:45:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 14:45:58.556304 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0582264-4be2-48db-82df-7dfc32479160 None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:58.591825 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14b0267-a80f-4a04-a26b-bedbd7f5fcb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:58.592716 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14b0267-a80f-4a04-a26b-bedbd7f5fcb0 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:58.593186 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14b0267-a80f-4a04-a26b-bedbd7f5fcb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:58.593672 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14b0267-a80f-4a04-a26b-bedbd7f5fcb0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:58.618080 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14b0267-a80f-4a04-a26b-bedbd7f5fcb0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:58.619525 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14b0267-a80f-4a04-a26b-bedbd7f5fcb0 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:58.638583 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application [None req-c14b0267-a80f-4a04-a26b-bedbd7f5fcb0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 14:45:58.638583 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:45:58.638583 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:45:58.638583 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:45:58.638583 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:45:58.638583 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:45:58.638583 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:45:58.638583 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:45:58.638583 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:45:58.638583 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:45:58.638583 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:45:58.638583 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:45:58.638583 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 14:45:58.638583 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 14:45:58.638583 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 14:45:58.638583 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 14:45:58.639573 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:45:58.639573 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:45:58.639573 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:45:58.639573 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:45:58.639573 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:45:58.639573 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:45:58.639573 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:45:58.639573 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application with Lock( Sep 01 14:45:58.639573 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:45:58.639573 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:45:58.639573 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:45:58.639573 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:45:58.639573 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:45:58.639573 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:45:58.639573 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:45:58.639573 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:45:58.639573 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 14:45:58.640477 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 14:45:58.640477 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 14:45:58.640477 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Sep 01 14:45:58.640869 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 221/462] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 1 14:45:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:45:58.646479 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9abea1-2b8d-4363-83b1-7a6ec8c9b14e None None] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:58.646479 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9abea1-2b8d-4363-83b1-7a6ec8c9b14e None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:58.646479 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9abea1-2b8d-4363-83b1-7a6ec8c9b14e None None] PATH_INFO: `/` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:58.646479 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 222/463] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 1 14:45:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:45:58.647534 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9abea1-2b8d-4363-83b1-7a6ec8c9b14e None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:58.677596 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0582264-4be2-48db-82df-7dfc32479160 None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:58.725834 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c65fc4-4f03-47e5-972b-4945a79897c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:58.726830 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c65fc4-4f03-47e5-972b-4945a79897c1 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:58.727329 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c65fc4-4f03-47e5-972b-4945a79897c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:58.727810 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c65fc4-4f03-47e5-972b-4945a79897c1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:58.742112 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c65fc4-4f03-47e5-972b-4945a79897c1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:58.744143 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c65fc4-4f03-47e5-972b-4945a79897c1 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:58.754984 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9abea1-2b8d-4363-83b1-7a6ec8c9b14e None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:58.764756 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application [None req-65c65fc4-4f03-47e5-972b-4945a79897c1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 14:45:58.764756 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:45:58.764756 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:45:58.764756 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:45:58.764756 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:45:58.764756 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:45:58.764756 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:45:58.764756 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:45:58.764756 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:45:58.764756 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:45:58.764756 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:45:58.764756 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:45:58.764756 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 14:45:58.764756 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 14:45:58.764756 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 14:45:58.764756 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 14:45:58.765664 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:45:58.765664 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:45:58.765664 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:45:58.765664 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:45:58.765664 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:45:58.765664 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:45:58.765664 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:45:58.765664 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application with Lock( Sep 01 14:45:58.765664 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:45:58.765664 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:45:58.765664 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:45:58.765664 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:45:58.765664 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:45:58.765664 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:45:58.765664 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:45:58.765664 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:45:58.765664 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 14:45:58.766592 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 14:45:58.766592 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 14:45:58.766592 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Sep 01 14:45:58.766592 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:58.771652 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 242/464] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 1 14:45:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 14:45:58.771652 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdea506a-1753-4c98-84cb-b16517c1aee9 None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:58.771652 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdea506a-1753-4c98-84cb-b16517c1aee9 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:58.771652 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdea506a-1753-4c98-84cb-b16517c1aee9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:58.774429 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:58.801689 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478a200c-e776-427f-a9fd-d651e2f9f31b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:58.802502 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478a200c-e776-427f-a9fd-d651e2f9f31b None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:58.802683 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478a200c-e776-427f-a9fd-d651e2f9f31b None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:58.802889 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478a200c-e776-427f-a9fd-d651e2f9f31b None admin] PATH_INFO: `/v3/domains` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:58.803529 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478a200c-e776-427f-a9fd-d651e2f9f31b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81341) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:45:58.803831 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478a200c-e776-427f-a9fd-d651e2f9f31b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:58.805225 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478a200c-e776-427f-a9fd-d651e2f9f31b None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:58.806961 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-fdea506a-1753-4c98-84cb-b16517c1aee9 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:45:58.809026 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-fdea506a-1753-4c98-84cb-b16517c1aee9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:45:58.822341 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 223/465] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Sep 1 14:45:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:45:58.822341 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478a200c-e776-427f-a9fd-d651e2f9f31b None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:58.888440 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 243/466] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 1 14:45:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 14:45:58.889907 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdea506a-1753-4c98-84cb-b16517c1aee9 None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:58.939174 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478a200c-e776-427f-a9fd-d651e2f9f31b None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:58.975205 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49993529-0a6f-4939-aa72-6477438c1be5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:58.976145 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49993529-0a6f-4939-aa72-6477438c1be5 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:58.976625 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49993529-0a6f-4939-aa72-6477438c1be5 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:58.977116 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49993529-0a6f-4939-aa72-6477438c1be5 None admin] PATH_INFO: `/v3/domains` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:58.977950 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49993529-0a6f-4939-aa72-6477438c1be5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81341) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:45:58.978495 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49993529-0a6f-4939-aa72-6477438c1be5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:58.980083 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49993529-0a6f-4939-aa72-6477438c1be5 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:58.992159 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 224/467] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Sep 1 14:45:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:45:58.997659 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49993529-0a6f-4939-aa72-6477438c1be5 None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:59.005557 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdea506a-1753-4c98-84cb-b16517c1aee9 None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:59.031365 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:59.035001 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:59.047458 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116c7b22-75c0-41f2-a891-8f9338ab50e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:59.048069 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116c7b22-75c0-41f2-a891-8f9338ab50e6 None admin] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:59.048348 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116c7b22-75c0-41f2-a891-8f9338ab50e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:59.048898 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116c7b22-75c0-41f2-a891-8f9338ab50e6 None admin] PATH_INFO: `/v3/users` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:59.049356 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116c7b22-75c0-41f2-a891-8f9338ab50e6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:59.050851 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116c7b22-75c0-41f2-a891-8f9338ab50e6 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:59.086266 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 244/468] 10.222.0.33 () {60 vars in 1132 bytes} [Thu Sep 1 14:45:58 2022] POST /identity/v3/users => generated 246 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:45:59.086266 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116c7b22-75c0-41f2-a891-8f9338ab50e6 None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:59.159341 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49993529-0a6f-4939-aa72-6477438c1be5 None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:59.217733 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8175eca8-6e48-4caa-be4f-fd9e5f9b7a77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:59.217733 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8175eca8-6e48-4caa-be4f-fd9e5f9b7a77 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:59.217733 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8175eca8-6e48-4caa-be4f-fd9e5f9b7a77 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:59.217733 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8175eca8-6e48-4caa-be4f-fd9e5f9b7a77 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:59.231063 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8175eca8-6e48-4caa-be4f-fd9e5f9b7a77 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:59.232610 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8175eca8-6e48-4caa-be4f-fd9e5f9b7a77 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:59.247789 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application [None req-8175eca8-6e48-4caa-be4f-fd9e5f9b7a77 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 14:45:59.247789 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:45:59.247789 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:45:59.247789 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:45:59.247789 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:45:59.247789 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:45:59.247789 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:45:59.247789 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:45:59.247789 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:45:59.247789 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:45:59.247789 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:45:59.247789 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:45:59.247789 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 14:45:59.247789 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 14:45:59.247789 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 14:45:59.247789 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 14:45:59.248757 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:45:59.248757 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:45:59.248757 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:45:59.248757 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:45:59.248757 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:45:59.248757 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:45:59.248757 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:45:59.248757 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application with Lock( Sep 01 14:45:59.248757 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:45:59.248757 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:45:59.248757 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:45:59.248757 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:45:59.248757 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:45:59.248757 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:45:59.248757 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:45:59.248757 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:45:59.248757 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 14:45:59.249654 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 14:45:59.249654 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 14:45:59.249654 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Sep 01 14:45:59.249654 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 225/469] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 1 14:45:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:45:59.250335 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8175eca8-6e48-4caa-be4f-fd9e5f9b7a77 None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:59.253016 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116c7b22-75c0-41f2-a891-8f9338ab50e6 None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:59.291588 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901892b9-29c3-4eca-984a-b25e62aaa5fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:59.292208 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901892b9-29c3-4eca-984a-b25e62aaa5fb None admin] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:59.292491 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901892b9-29c3-4eca-984a-b25e62aaa5fb None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:59.292762 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901892b9-29c3-4eca-984a-b25e62aaa5fb None admin] PATH_INFO: `/v3/users` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:59.293408 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901892b9-29c3-4eca-984a-b25e62aaa5fb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:59.295669 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901892b9-29c3-4eca-984a-b25e62aaa5fb None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:59.321813 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 245/470] 10.222.0.33 () {60 vars in 1132 bytes} [Thu Sep 1 14:45:59 2022] POST /identity/v3/users => generated 248 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:45:59.326799 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901892b9-29c3-4eca-984a-b25e62aaa5fb None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:59.347394 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:59.386285 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8175eca8-6e48-4caa-be4f-fd9e5f9b7a77 None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:59.390390 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:59.438662 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f34de7f-27eb-4311-ae3d-6fbdd2249c5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:59.439211 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f34de7f-27eb-4311-ae3d-6fbdd2249c5f None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:59.439489 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f34de7f-27eb-4311-ae3d-6fbdd2249c5f None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:59.439756 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f34de7f-27eb-4311-ae3d-6fbdd2249c5f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:59.453293 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f34de7f-27eb-4311-ae3d-6fbdd2249c5f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:59.454846 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f34de7f-27eb-4311-ae3d-6fbdd2249c5f None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:59.471325 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application [None req-7f34de7f-27eb-4311-ae3d-6fbdd2249c5f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 14:45:59.471325 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:45:59.471325 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:45:59.471325 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:45:59.471325 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:45:59.471325 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:45:59.471325 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:45:59.471325 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:45:59.471325 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:45:59.471325 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:45:59.471325 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:45:59.471325 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:45:59.471325 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 14:45:59.471325 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 14:45:59.471325 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 14:45:59.471325 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 14:45:59.472381 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:45:59.472381 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:45:59.472381 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:45:59.472381 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:45:59.472381 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:45:59.472381 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:45:59.472381 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:45:59.472381 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application with Lock( Sep 01 14:45:59.472381 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:45:59.472381 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:45:59.472381 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:45:59.472381 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:45:59.472381 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:45:59.472381 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:45:59.472381 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:45:59.472381 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:45:59.472381 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 14:45:59.473297 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 14:45:59.473297 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 14:45:59.473297 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Sep 01 14:45:59.473297 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 226/471] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 1 14:45:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:45:59.474208 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2365e80-b5d2-429b-8da8-6e0ef7135752 None None] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:59.474548 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2365e80-b5d2-429b-8da8-6e0ef7135752 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:59.474815 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2365e80-b5d2-429b-8da8-6e0ef7135752 None None] PATH_INFO: `/` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:59.476167 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 227/472] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 1 14:45:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:45:59.481434 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2365e80-b5d2-429b-8da8-6e0ef7135752 None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:59.499091 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901892b9-29c3-4eca-984a-b25e62aaa5fb None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:59.544864 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6512832-b7b7-4fb6-b602-7882b046e6f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:59.545483 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6512832-b7b7-4fb6-b602-7882b046e6f9 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:59.545763 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6512832-b7b7-4fb6-b602-7882b046e6f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:59.546030 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6512832-b7b7-4fb6-b602-7882b046e6f9 None admin] PATH_INFO: `/v3/domains` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:59.546721 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6512832-b7b7-4fb6-b602-7882b046e6f9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:45:59.547027 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6512832-b7b7-4fb6-b602-7882b046e6f9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:59.548403 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6512832-b7b7-4fb6-b602-7882b046e6f9 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:59.569362 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 246/473] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Sep 1 14:45:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:45:59.571004 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3dbbdb-bf21-4170-a7e9-726c20a4d6cd None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:59.571287 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3dbbdb-bf21-4170-a7e9-726c20a4d6cd None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:59.571554 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3dbbdb-bf21-4170-a7e9-726c20a4d6cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:59.589596 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-ff3dbbdb-bf21-4170-a7e9-726c20a4d6cd None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:45:59.590809 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-ff3dbbdb-bf21-4170-a7e9-726c20a4d6cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:45:59.612651 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2365e80-b5d2-429b-8da8-6e0ef7135752 None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:59.658503 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0713f75-25f1-42fe-ac99-43fdba6f379f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:59.659635 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0713f75-25f1-42fe-ac99-43fdba6f379f None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:59.663241 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0713f75-25f1-42fe-ac99-43fdba6f379f None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:59.663672 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0713f75-25f1-42fe-ac99-43fdba6f379f None admin] PATH_INFO: `/v3/domains` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:59.664549 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0713f75-25f1-42fe-ac99-43fdba6f379f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81341) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:45:59.665020 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0713f75-25f1-42fe-ac99-43fdba6f379f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:59.665794 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 247/474] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 1 14:45:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 14:45:59.666784 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0713f75-25f1-42fe-ac99-43fdba6f379f None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:59.666901 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3dbbdb-bf21-4170-a7e9-726c20a4d6cd None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:59.676001 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:59.683319 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 228/475] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Sep 1 14:45:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:45:59.689768 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0713f75-25f1-42fe-ac99-43fdba6f379f None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:45:59.715709 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:45:59.832505 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0713f75-25f1-42fe-ac99-43fdba6f379f None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:59.868949 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3dbbdb-bf21-4170-a7e9-726c20a4d6cd None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:45:59.880747 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0e5f1a-946e-45c5-9522-0b2e9076fd71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:59.881709 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0e5f1a-946e-45c5-9522-0b2e9076fd71 None admin] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:59.882188 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0e5f1a-946e-45c5-9522-0b2e9076fd71 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:59.882702 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0e5f1a-946e-45c5-9522-0b2e9076fd71 None admin] PATH_INFO: `/v3/users` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:59.883541 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0e5f1a-946e-45c5-9522-0b2e9076fd71 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:59.885182 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0e5f1a-946e-45c5-9522-0b2e9076fd71 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:59.919921 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 229/476] 10.222.0.33 () {60 vars in 1132 bytes} [Thu Sep 1 14:45:59 2022] POST /identity/v3/users => generated 249 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:45:59.941405 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed483d7-0d45-44c7-bad8-83830a7a4a97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:45:59.942847 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed483d7-0d45-44c7-bad8-83830a7a4a97 None admin] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:45:59.943330 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed483d7-0d45-44c7-bad8-83830a7a4a97 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:45:59.943800 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed483d7-0d45-44c7-bad8-83830a7a4a97 None admin] PATH_INFO: `/v3/users` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:45:59.944671 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed483d7-0d45-44c7-bad8-83830a7a4a97 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:45:59.946715 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed483d7-0d45-44c7-bad8-83830a7a4a97 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:45:59.972460 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 248/477] 10.222.0.33 () {60 vars in 1132 bytes} [Thu Sep 1 14:45:59 2022] POST /identity/v3/users => generated 248 bytes in 306 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:45:59.980809 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0e5f1a-946e-45c5-9522-0b2e9076fd71 None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:00.043416 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:00.105211 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0e5f1a-946e-45c5-9522-0b2e9076fd71 None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:00.141694 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8468b844-dc45-47ff-b0dd-137953ab1482 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:00.142335 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8468b844-dc45-47ff-b0dd-137953ab1482 None admin] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:00.142610 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8468b844-dc45-47ff-b0dd-137953ab1482 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:00.142876 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8468b844-dc45-47ff-b0dd-137953ab1482 None admin] PATH_INFO: `/v3/roles` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:00.143552 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8468b844-dc45-47ff-b0dd-137953ab1482 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:00.144945 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8468b844-dc45-47ff-b0dd-137953ab1482 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:00.156174 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.sql.core [None req-8468b844-dc45-47ff-b0dd-137953ab1482 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 01 14:46:00.156174 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 01 14:46:00.156174 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [parameters: {'id': '6a53e80d7853447b869cf6c44f8050ab', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 01 14:46:00.156174 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81341) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 01 14:46:00.173734 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application [None req-8468b844-dc45-47ff-b0dd-137953ab1482 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 01 14:46:00.173734 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:46:00.173734 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 01 14:46:00.173734 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 01 14:46:00.173734 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 01 14:46:00.173734 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 01 14:46:00.173734 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 01 14:46:00.173734 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application result = self._query(query) Sep 01 14:46:00.173734 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 01 14:46:00.173734 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application conn.query(q) Sep 01 14:46:00.173734 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 01 14:46:00.173734 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 01 14:46:00.173734 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 01 14:46:00.173734 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application result.read() Sep 01 14:46:00.173734 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 01 14:46:00.174720 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 01 14:46:00.174720 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 01 14:46:00.174720 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 01 14:46:00.174720 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 01 14:46:00.174720 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 01 14:46:00.174720 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 01 14:46:00.174720 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 01 14:46:00.174720 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 01 14:46:00.174720 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Sep 01 14:46:00.174720 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 01 14:46:00.174720 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Sep 01 14:46:00.174720 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:46:00.174720 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 01 14:46:00.174720 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 01 14:46:00.174720 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Sep 01 14:46:00.174720 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return ref.to_dict() Sep 01 14:46:00.174720 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 01 14:46:00.174720 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application next(self.gen) Sep 01 14:46:00.175700 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 01 14:46:00.175700 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application yield resource Sep 01 14:46:00.175700 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 01 14:46:00.175700 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application next(self.gen) Sep 01 14:46:00.175700 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 01 14:46:00.175700 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application self.session.rollback() Sep 01 14:46:00.175700 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 01 14:46:00.175700 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application self.force_reraise() Sep 01 14:46:00.175700 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 01 14:46:00.175700 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application raise self.value Sep 01 14:46:00.175700 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 01 14:46:00.175700 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 01 14:46:00.175700 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 01 14:46:00.175700 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application session.commit() Sep 01 14:46:00.175700 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 01 14:46:00.175700 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 01 14:46:00.175700 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 01 14:46:00.176589 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application self._prepare_impl() Sep 01 14:46:00.176589 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 01 14:46:00.176589 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application self.session.flush() Sep 01 14:46:00.176589 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 01 14:46:00.176589 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application self._flush(objects) Sep 01 14:46:00.176589 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 01 14:46:00.176589 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 01 14:46:00.176589 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 01 14:46:00.176589 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application compat.raise_( Sep 01 14:46:00.176589 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 01 14:46:00.176589 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application raise exception Sep 01 14:46:00.176589 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 01 14:46:00.176589 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application flush_context.execute() Sep 01 14:46:00.176589 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 01 14:46:00.176589 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application rec.execute(self) Sep 01 14:46:00.176589 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 01 14:46:00.176589 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 01 14:46:00.177591 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 01 14:46:00.177591 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 01 14:46:00.177591 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 01 14:46:00.177591 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 01 14:46:00.177591 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 01 14:46:00.177591 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 01 14:46:00.177591 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 01 14:46:00.177591 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 01 14:46:00.177591 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 01 14:46:00.177591 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 01 14:46:00.177591 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 01 14:46:00.177591 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 01 14:46:00.177591 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 01 14:46:00.177591 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 01 14:46:00.177591 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 01 14:46:00.177591 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application raise exception Sep 01 14:46:00.179713 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 01 14:46:00.179713 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 01 14:46:00.179713 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 01 14:46:00.179713 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 01 14:46:00.179713 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 01 14:46:00.179713 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application result = self._query(query) Sep 01 14:46:00.179713 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 01 14:46:00.179713 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application conn.query(q) Sep 01 14:46:00.179713 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 01 14:46:00.179713 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 01 14:46:00.179713 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 01 14:46:00.179713 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application result.read() Sep 01 14:46:00.179713 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 01 14:46:00.179713 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 01 14:46:00.179713 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 01 14:46:00.179713 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 01 14:46:00.181889 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 01 14:46:00.181889 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 01 14:46:00.181889 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 01 14:46:00.181889 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 01 14:46:00.181889 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 01 14:46:00.181889 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 01 14:46:00.181889 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application [parameters: {'id': '6a53e80d7853447b869cf6c44f8050ab', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 01 14:46:00.181889 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 01 14:46:00.181889 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Sep 01 14:46:00.181889 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 14:46:00.181889 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Sep 01 14:46:00.181889 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:46:00.181889 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:46:00.181889 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:46:00.181889 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:46:00.181889 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:46:00.184159 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:46:00.184159 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:46:00.184159 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:46:00.184159 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:46:00.184159 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:46:00.184159 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:46:00.184159 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Sep 01 14:46:00.184159 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Sep 01 14:46:00.184159 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:46:00.184159 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:46:00.184159 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Sep 01 14:46:00.184159 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Sep 01 14:46:00.184159 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 01 14:46:00.184159 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 01 14:46:00.184159 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 01 14:46:00.184159 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Sep 01 14:46:00.186545 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 230/478] 10.222.0.33 () {60 vars in 1131 bytes} [Thu Sep 1 14:45:59 2022] POST /identity/v3/roles => generated 144 bytes in 195 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:46:00.186615 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed483d7-0d45-44c7-bad8-83830a7a4a97 None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:00.278318 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:00.299978 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed483d7-0d45-44c7-bad8-83830a7a4a97 None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:00.336440 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c031fbde-3267-4f62-936c-94184df16334 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:00.337078 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c031fbde-3267-4f62-936c-94184df16334 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:00.337351 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c031fbde-3267-4f62-936c-94184df16334 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:00.337618 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c031fbde-3267-4f62-936c-94184df16334 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:00.349688 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c031fbde-3267-4f62-936c-94184df16334 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 14:46:00.354556 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application [None req-c031fbde-3267-4f62-936c-94184df16334 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 14:46:00.354556 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:46:00.354556 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:46:00.354556 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:46:00.354556 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:46:00.354556 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:46:00.354556 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:46:00.354556 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:46:00.354556 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:46:00.354556 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:46:00.354556 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:46:00.354556 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:46:00.354556 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 14:46:00.354556 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 14:46:00.354556 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 14:46:00.354556 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 14:46:00.355457 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 14:46:00.355457 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 14:46:00.355457 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 14:46:00.355457 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 14:46:00.355457 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:46:00.355457 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:46:00.355457 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:46:00.355457 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:46:00.355457 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:46:00.355457 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:46:00.355457 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:46:00.355457 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application with Lock( Sep 01 14:46:00.355457 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:46:00.355457 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:46:00.355457 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:46:00.355457 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:46:00.356288 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:46:00.356288 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:46:00.356288 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:46:00.356288 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:46:00.356288 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 14:46:00.356288 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 14:46:00.356288 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 14:46:00.356288 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 14:46:00.356288 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 14:46:00.356288 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 14:46:00.356288 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 14:46:00.356288 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Sep 01 14:46:00.360413 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 249/479] 10.222.0.33 () {56 vars in 1114 bytes} [Thu Sep 1 14:46:00 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 14:46:00.361837 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8468b844-dc45-47ff-b0dd-137953ab1482 None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:00.469926 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8468b844-dc45-47ff-b0dd-137953ab1482 None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:00.486608 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:00.504906 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea83de3b-5df0-450d-8e00-fc817c222213 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:00.506259 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea83de3b-5df0-450d-8e00-fc817c222213 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:00.506583 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea83de3b-5df0-450d-8e00-fc817c222213 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:00.507205 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea83de3b-5df0-450d-8e00-fc817c222213 None admin] PATH_INFO: `/v3/roles` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:00.508233 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea83de3b-5df0-450d-8e00-fc817c222213 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81341) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:46:00.508914 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea83de3b-5df0-450d-8e00-fc817c222213 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:00.510804 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea83de3b-5df0-450d-8e00-fc817c222213 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:00.522487 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 231/480] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Sep 1 14:46:00 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:46:01.396544 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2437e21-3b2b-49e6-b55e-d46bc4c03376 None None] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:01.397086 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2437e21-3b2b-49e6-b55e-d46bc4c03376 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:01.397086 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2437e21-3b2b-49e6-b55e-d46bc4c03376 None None] PATH_INFO: `/` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:01.399341 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 250/481] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 1 14:46:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:46:01.404357 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8db7cc9-5dc3-4ac4-b4a8-7f9a69a54504 None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:01.405136 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8db7cc9-5dc3-4ac4-b4a8-7f9a69a54504 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:01.405618 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8db7cc9-5dc3-4ac4-b4a8-7f9a69a54504 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:01.434551 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-a8db7cc9-5dc3-4ac4-b4a8-7f9a69a54504 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:46:01.436103 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-a8db7cc9-5dc3-4ac4-b4a8-7f9a69a54504 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:46:01.509416 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 232/482] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 1 14:46:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 14:46:01.684628 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469c6bc7-1e32-4563-b3d9-6a051e8a287e None None] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:01.685269 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469c6bc7-1e32-4563-b3d9-6a051e8a287e None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:01.685730 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469c6bc7-1e32-4563-b3d9-6a051e8a287e None None] PATH_INFO: `/` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:01.687362 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 251/483] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 1 14:46:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:46:01.692766 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2220dc-0117-4b80-a996-1ec5d9b75a0f None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:01.693295 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2220dc-0117-4b80-a996-1ec5d9b75a0f None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:01.693782 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2220dc-0117-4b80-a996-1ec5d9b75a0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:01.720879 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-5f2220dc-0117-4b80-a996-1ec5d9b75a0f None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:46:01.722399 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-5f2220dc-0117-4b80-a996-1ec5d9b75a0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:46:01.808791 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 233/484] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 1 14:46:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 14:46:01.874529 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469c6bc7-1e32-4563-b3d9-6a051e8a287e None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:01.981363 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469c6bc7-1e32-4563-b3d9-6a051e8a287e None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:02.018228 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812a69f8-6abf-4c53-9ad8-e15acf0afcc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:02.019258 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812a69f8-6abf-4c53-9ad8-e15acf0afcc9 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:02.019785 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812a69f8-6abf-4c53-9ad8-e15acf0afcc9 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:02.020242 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812a69f8-6abf-4c53-9ad8-e15acf0afcc9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:02.032091 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812a69f8-6abf-4c53-9ad8-e15acf0afcc9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 14:46:02.037132 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application [None req-812a69f8-6abf-4c53-9ad8-e15acf0afcc9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 14:46:02.037132 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:46:02.037132 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:46:02.037132 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:46:02.037132 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:46:02.037132 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:46:02.037132 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:46:02.037132 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:46:02.037132 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:46:02.037132 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:46:02.037132 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:46:02.037132 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:46:02.037132 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 14:46:02.037132 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 14:46:02.037132 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 14:46:02.037132 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 14:46:02.038033 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 14:46:02.038033 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 14:46:02.038033 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 14:46:02.038033 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 14:46:02.038033 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:46:02.038033 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:46:02.038033 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:46:02.038033 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:46:02.038033 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:46:02.038033 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:46:02.038033 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:46:02.038033 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application with Lock( Sep 01 14:46:02.038033 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:46:02.038033 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:46:02.038033 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:46:02.038033 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:46:02.038891 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:46:02.038891 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:46:02.038891 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:46:02.038891 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:46:02.038891 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 14:46:02.038891 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 14:46:02.038891 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 14:46:02.038891 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 14:46:02.038891 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 14:46:02.038891 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 14:46:02.038891 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 14:46:02.038891 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Sep 01 14:46:02.041435 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 252/485] 10.222.0.33 () {56 vars in 1102 bytes} [Thu Sep 1 14:46:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:46:02.047310 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2220dc-0117-4b80-a996-1ec5d9b75a0f None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:02.082129 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:02.161903 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2220dc-0117-4b80-a996-1ec5d9b75a0f None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:02.195577 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59185c43-eecd-4e66-9536-eceea926bee8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:02.196440 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59185c43-eecd-4e66-9536-eceea926bee8 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:02.196717 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59185c43-eecd-4e66-9536-eceea926bee8 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:02.197045 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59185c43-eecd-4e66-9536-eceea926bee8 None admin] PATH_INFO: `/v3/roles` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:02.197712 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59185c43-eecd-4e66-9536-eceea926bee8 None admin] RBAC: Adding query filter params (name=service) {{(pid=81341) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:46:02.197906 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812a69f8-6abf-4c53-9ad8-e15acf0afcc9 None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:02.198015 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59185c43-eecd-4e66-9536-eceea926bee8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:02.199420 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59185c43-eecd-4e66-9536-eceea926bee8 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:02.201651 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:02.210660 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 234/486] 10.222.0.33 () {56 vars in 1103 bytes} [Thu Sep 1 14:46:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:46:02.216550 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59185c43-eecd-4e66-9536-eceea926bee8 None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:02.370644 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:02.378814 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59185c43-eecd-4e66-9536-eceea926bee8 None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:02.394994 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812a69f8-6abf-4c53-9ad8-e15acf0afcc9 None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:02.438379 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08544e37-2c64-4aef-af32-a150c3c5d356 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:02.439039 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08544e37-2c64-4aef-af32-a150c3c5d356 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:02.439282 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08544e37-2c64-4aef-af32-a150c3c5d356 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:02.439553 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08544e37-2c64-4aef-af32-a150c3c5d356 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:02.457753 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08544e37-2c64-4aef-af32-a150c3c5d356 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:02.459305 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08544e37-2c64-4aef-af32-a150c3c5d356 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:02.461195 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7fb96a-1d1e-4d3f-bb9b-81b29cff185a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:02.461885 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7fb96a-1d1e-4d3f-bb9b-81b29cff185a None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:02.462174 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7fb96a-1d1e-4d3f-bb9b-81b29cff185a None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:02.462469 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7fb96a-1d1e-4d3f-bb9b-81b29cff185a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:02.481183 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7fb96a-1d1e-4d3f-bb9b-81b29cff185a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 14:46:02.490365 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application [None req-2f7fb96a-1d1e-4d3f-bb9b-81b29cff185a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 14:46:02.490365 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:46:02.490365 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:46:02.490365 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:46:02.490365 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:46:02.490365 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:46:02.490365 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:46:02.490365 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:46:02.490365 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:46:02.490365 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:46:02.490365 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:46:02.490365 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:46:02.490365 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 14:46:02.490365 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 14:46:02.490365 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 14:46:02.490365 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 14:46:02.491317 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 14:46:02.491317 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 14:46:02.491317 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 14:46:02.491317 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 14:46:02.491317 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:46:02.491317 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:46:02.491317 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:46:02.491317 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:46:02.491317 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:46:02.491317 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:46:02.491317 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:46:02.491317 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application with Lock( Sep 01 14:46:02.491317 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:46:02.491317 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:46:02.491317 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:46:02.491317 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:46:02.492519 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:46:02.492519 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:46:02.492519 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:46:02.492519 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:46:02.492519 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 14:46:02.492519 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 14:46:02.492519 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 14:46:02.492519 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 14:46:02.492519 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 14:46:02.492519 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 14:46:02.492519 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 14:46:02.492519 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Sep 01 14:46:02.492519 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 253/487] 10.222.0.33 () {56 vars in 1102 bytes} [Thu Sep 1 14:46:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 14:46:02.493433 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application [None req-08544e37-2c64-4aef-af32-a150c3c5d356 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 14:46:02.493433 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:46:02.493433 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:46:02.493433 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:46:02.493433 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:46:02.493433 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:46:02.493433 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:46:02.493433 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:46:02.493433 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:46:02.493433 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:46:02.493433 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:46:02.493433 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:46:02.493433 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 14:46:02.493433 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 14:46:02.493433 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 14:46:02.493433 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 14:46:02.494380 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:46:02.494380 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:46:02.494380 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:46:02.494380 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:46:02.494380 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:46:02.494380 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:46:02.494380 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:46:02.494380 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application with Lock( Sep 01 14:46:02.494380 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:46:02.494380 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:46:02.494380 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:46:02.494380 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:46:02.494380 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:46:02.494380 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:46:02.494380 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:46:02.494380 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:46:02.494380 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 14:46:02.496286 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 14:46:02.496286 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 14:46:02.496286 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Sep 01 14:46:02.496286 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 235/488] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 1 14:46:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 14:46:02.503603 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7fb96a-1d1e-4d3f-bb9b-81b29cff185a None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:02.507109 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08544e37-2c64-4aef-af32-a150c3c5d356 None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:02.585018 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:02.635527 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7fb96a-1d1e-4d3f-bb9b-81b29cff185a None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:02.643764 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08544e37-2c64-4aef-af32-a150c3c5d356 None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:02.687889 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f72543e-7fcc-468f-b9e5-acd5f9595831 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:02.689462 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f72543e-7fcc-468f-b9e5-acd5f9595831 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:02.690072 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f72543e-7fcc-468f-b9e5-acd5f9595831 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:02.690599 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f72543e-7fcc-468f-b9e5-acd5f9595831 None admin] PATH_INFO: `/v3/domains` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:02.691438 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f72543e-7fcc-468f-b9e5-acd5f9595831 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:46:02.691877 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f72543e-7fcc-468f-b9e5-acd5f9595831 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:02.693449 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f72543e-7fcc-468f-b9e5-acd5f9595831 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:02.694551 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee60af0-7dd0-46f7-a09c-78da4e609314 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:02.695435 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee60af0-7dd0-46f7-a09c-78da4e609314 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:02.696067 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee60af0-7dd0-46f7-a09c-78da4e609314 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:02.696450 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee60af0-7dd0-46f7-a09c-78da4e609314 None admin] PATH_INFO: `/v3/roles` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:02.697670 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee60af0-7dd0-46f7-a09c-78da4e609314 None admin] RBAC: Adding query filter params (name=service) {{(pid=81341) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:46:02.698074 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee60af0-7dd0-46f7-a09c-78da4e609314 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:02.699596 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee60af0-7dd0-46f7-a09c-78da4e609314 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:02.708327 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 254/489] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Sep 1 14:46:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:46:02.709791 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ae4e7d-eebd-4892-9878-b6b45a998efd None None] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:02.710221 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ae4e7d-eebd-4892-9878-b6b45a998efd None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:02.710639 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ae4e7d-eebd-4892-9878-b6b45a998efd None None] PATH_INFO: `/` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:02.716120 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 255/490] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 1 14:46:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:46:02.716120 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ae4e7d-eebd-4892-9878-b6b45a998efd None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:02.720637 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 236/491] 10.222.0.33 () {56 vars in 1103 bytes} [Thu Sep 1 14:46:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:46:02.721768 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194b2ac0-bfc1-4f62-b884-760699607b8f None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:02.722052 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194b2ac0-bfc1-4f62-b884-760699607b8f None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:02.722347 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194b2ac0-bfc1-4f62-b884-760699607b8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:02.745783 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-194b2ac0-bfc1-4f62-b884-760699607b8f None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:46:02.747353 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-194b2ac0-bfc1-4f62-b884-760699607b8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:46:02.757683 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:02.830175 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 237/492] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 1 14:46:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 14:46:02.830985 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194b2ac0-bfc1-4f62-b884-760699607b8f None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:02.847981 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ae4e7d-eebd-4892-9878-b6b45a998efd None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:02.872256 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:02.884862 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a18bcdc-c798-43bc-8825-49b876d1a328 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:02.885485 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a18bcdc-c798-43bc-8825-49b876d1a328 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:02.885895 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a18bcdc-c798-43bc-8825-49b876d1a328 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:02.886066 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a18bcdc-c798-43bc-8825-49b876d1a328 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:02.894947 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a18bcdc-c798-43bc-8825-49b876d1a328 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:02.896438 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a18bcdc-c798-43bc-8825-49b876d1a328 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:02.909470 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application [None req-7a18bcdc-c798-43bc-8825-49b876d1a328 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 01 14:46:02.909470 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:46:02.909470 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:46:02.909470 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:46:02.909470 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:46:02.909470 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:46:02.909470 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:46:02.909470 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:46:02.909470 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:46:02.909470 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:46:02.909470 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:46:02.909470 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:46:02.909470 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 14:46:02.909470 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 14:46:02.909470 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 14:46:02.909470 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 14:46:02.910458 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:46:02.910458 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:46:02.910458 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 14:46:02.910458 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 14:46:02.910458 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 14:46:02.910458 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 14:46:02.910458 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:46:02.910458 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:46:02.910458 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:46:02.910458 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:46:02.910458 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:46:02.910458 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application with Lock( Sep 01 14:46:02.910458 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:46:02.910458 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:46:02.910458 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:46:02.910458 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:46:02.910458 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:46:02.910458 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:46:02.911910 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:46:02.911910 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:46:02.911910 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 14:46:02.911910 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 14:46:02.911910 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 14:46:02.911910 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 14:46:02.911910 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 14:46:02.911910 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 14:46:02.911910 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 01 14:46:02.911910 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Sep 01 14:46:02.911910 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 256/493] 10.222.0.33 () {56 vars in 1093 bytes} [Thu Sep 1 14:46:02 2022] GET /identity/v3/users/nova => generated 82 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:46:02.912983 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b809f2-f05c-4427-a2e9-c8ea81146004 None None] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:02.913078 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b809f2-f05c-4427-a2e9-c8ea81146004 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:02.913299 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b809f2-f05c-4427-a2e9-c8ea81146004 None None] PATH_INFO: `/` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:02.918799 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 257/494] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 1 14:46:02 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:46:02.919597 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b809f2-f05c-4427-a2e9-c8ea81146004 None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:02.968313 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194b2ac0-bfc1-4f62-b884-760699607b8f None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:03.012874 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f65657-6131-4350-9702-3bfe0e20b17e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:03.013501 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f65657-6131-4350-9702-3bfe0e20b17e None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:03.013824 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f65657-6131-4350-9702-3bfe0e20b17e None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:03.014087 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f65657-6131-4350-9702-3bfe0e20b17e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:03.025338 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f65657-6131-4350-9702-3bfe0e20b17e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:03.026855 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f65657-6131-4350-9702-3bfe0e20b17e None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:03.028798 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:03.029778 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b809f2-f05c-4427-a2e9-c8ea81146004 None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:03.043761 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application [None req-22f65657-6131-4350-9702-3bfe0e20b17e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 14:46:03.043761 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:46:03.043761 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:46:03.043761 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:46:03.043761 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:46:03.043761 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:46:03.043761 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:46:03.043761 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:46:03.043761 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:46:03.043761 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:46:03.043761 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:46:03.043761 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:46:03.043761 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 14:46:03.043761 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 14:46:03.043761 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 14:46:03.043761 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 14:46:03.044855 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:46:03.044855 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:46:03.044855 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:46:03.044855 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:46:03.044855 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:46:03.044855 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:46:03.044855 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:46:03.044855 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application with Lock( Sep 01 14:46:03.044855 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:46:03.044855 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:46:03.044855 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:46:03.044855 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:46:03.044855 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:46:03.044855 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:46:03.044855 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:46:03.044855 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:46:03.044855 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 14:46:03.045757 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 14:46:03.045757 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 14:46:03.045757 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Sep 01 14:46:03.045757 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 238/495] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 1 14:46:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:46:03.046739 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b852ed05-1b9c-4211-9703-326331fd7e4d None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:03.047022 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b852ed05-1b9c-4211-9703-326331fd7e4d None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:03.047296 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b852ed05-1b9c-4211-9703-326331fd7e4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:03.064857 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-b852ed05-1b9c-4211-9703-326331fd7e4d None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:46:03.066046 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-b852ed05-1b9c-4211-9703-326331fd7e4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:46:03.072855 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62450991-2944-42f8-9b1d-ec3b3a518945 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:03.073852 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62450991-2944-42f8-9b1d-ec3b3a518945 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:03.074132 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62450991-2944-42f8-9b1d-ec3b3a518945 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:03.074434 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62450991-2944-42f8-9b1d-ec3b3a518945 None admin] PATH_INFO: `/v3/users` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:03.075161 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62450991-2944-42f8-9b1d-ec3b3a518945 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:46:03.075869 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62450991-2944-42f8-9b1d-ec3b3a518945 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:03.077641 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62450991-2944-42f8-9b1d-ec3b3a518945 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:03.121824 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 258/496] 10.222.0.33 () {56 vars in 1133 bytes} [Thu Sep 1 14:46:02 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:46:03.123148 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fa6234-8b3b-4142-806f-40e931fb871e None None] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:03.123432 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fa6234-8b3b-4142-806f-40e931fb871e None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:03.123695 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fa6234-8b3b-4142-806f-40e931fb871e None None] PATH_INFO: `/` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:03.124864 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 259/497] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 1 14:46:03 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:46:03.125537 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fa6234-8b3b-4142-806f-40e931fb871e None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:03.146336 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 239/498] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 1 14:46:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 14:46:03.146336 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b852ed05-1b9c-4211-9703-326331fd7e4d None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:03.172030 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:03.240413 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fa6234-8b3b-4142-806f-40e931fb871e None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:03.248335 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b852ed05-1b9c-4211-9703-326331fd7e4d None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:03.272643 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:03.280204 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d1ec31-e1a4-4ddd-a11e-a5bec45b0aa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:03.280806 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d1ec31-e1a4-4ddd-a11e-a5bec45b0aa4 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:03.281089 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d1ec31-e1a4-4ddd-a11e-a5bec45b0aa4 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:03.281361 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d1ec31-e1a4-4ddd-a11e-a5bec45b0aa4 None admin] PATH_INFO: `/v3/domains` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:03.282007 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d1ec31-e1a4-4ddd-a11e-a5bec45b0aa4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:46:03.282346 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d1ec31-e1a4-4ddd-a11e-a5bec45b0aa4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:03.283733 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d1ec31-e1a4-4ddd-a11e-a5bec45b0aa4 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:03.284537 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b2dc3b-7f7e-4650-81eb-6ae0681543d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:03.285443 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b2dc3b-7f7e-4650-81eb-6ae0681543d7 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:03.285725 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b2dc3b-7f7e-4650-81eb-6ae0681543d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:03.287010 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b2dc3b-7f7e-4650-81eb-6ae0681543d7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:03.295932 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 260/499] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Sep 1 14:46:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:46:03.296536 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d1ec31-e1a4-4ddd-a11e-a5bec45b0aa4 None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:03.301828 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b2dc3b-7f7e-4650-81eb-6ae0681543d7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 14:46:03.306610 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application [None req-87b2dc3b-7f7e-4650-81eb-6ae0681543d7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 14:46:03.306610 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:46:03.306610 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:46:03.306610 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:46:03.306610 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:46:03.306610 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:46:03.306610 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:46:03.306610 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:46:03.306610 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:46:03.306610 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:46:03.306610 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:46:03.306610 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:46:03.306610 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 14:46:03.306610 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 14:46:03.306610 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 14:46:03.306610 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 14:46:03.307666 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 14:46:03.307666 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 14:46:03.307666 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 14:46:03.307666 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 14:46:03.307666 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:46:03.307666 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:46:03.307666 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:46:03.307666 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:46:03.307666 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:46:03.307666 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:46:03.307666 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:46:03.307666 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application with Lock( Sep 01 14:46:03.307666 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:46:03.307666 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:46:03.307666 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:46:03.307666 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:46:03.308507 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:46:03.308507 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:46:03.308507 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:46:03.308507 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:46:03.308507 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 14:46:03.308507 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 14:46:03.308507 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 14:46:03.308507 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 14:46:03.308507 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 14:46:03.308507 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 14:46:03.308507 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 14:46:03.308507 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Sep 01 14:46:03.308507 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 240/500] 10.222.0.33 () {56 vars in 1102 bytes} [Thu Sep 1 14:46:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:46:03.309195 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44364029-588d-4323-aff0-554784c0240e None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:03.310101 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44364029-588d-4323-aff0-554784c0240e None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:03.310101 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44364029-588d-4323-aff0-554784c0240e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:03.331078 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-44364029-588d-4323-aff0-554784c0240e None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:46:03.332286 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-44364029-588d-4323-aff0-554784c0240e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:46:03.406364 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 241/501] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 1 14:46:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 14:46:03.406364 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44364029-588d-4323-aff0-554784c0240e None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:03.413524 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:03.416441 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d1ec31-e1a4-4ddd-a11e-a5bec45b0aa4 None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:03.452410 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69c5120-5ef6-4c10-a27c-8d114c59bb0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:03.453003 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69c5120-5ef6-4c10-a27c-8d114c59bb0c None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:03.453275 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69c5120-5ef6-4c10-a27c-8d114c59bb0c None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:03.453539 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69c5120-5ef6-4c10-a27c-8d114c59bb0c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:03.513021 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44364029-588d-4323-aff0-554784c0240e None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:03.523700 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69c5120-5ef6-4c10-a27c-8d114c59bb0c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:03.525165 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69c5120-5ef6-4c10-a27c-8d114c59bb0c None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:03.529749 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:03.553430 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc1dca7-c367-47a1-ad29-346ef3a6643e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:03.554056 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc1dca7-c367-47a1-ad29-346ef3a6643e None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:03.554508 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc1dca7-c367-47a1-ad29-346ef3a6643e None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:03.554782 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc1dca7-c367-47a1-ad29-346ef3a6643e None admin] PATH_INFO: `/v3/users/glance` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:03.564047 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc1dca7-c367-47a1-ad29-346ef3a6643e None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:03.565844 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc1dca7-c367-47a1-ad29-346ef3a6643e None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:03.581896 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application [None req-cdc1dca7-c367-47a1-ad29-346ef3a6643e None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 01 14:46:03.581896 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:46:03.581896 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:46:03.581896 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:46:03.581896 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:46:03.581896 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:46:03.581896 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:46:03.581896 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:46:03.581896 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:46:03.581896 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:46:03.581896 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:46:03.581896 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:46:03.581896 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 14:46:03.581896 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 14:46:03.581896 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 14:46:03.581896 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 14:46:03.582851 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:46:03.582851 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:46:03.582851 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 14:46:03.582851 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 14:46:03.582851 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 14:46:03.582851 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 14:46:03.582851 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:46:03.582851 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:46:03.582851 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:46:03.582851 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:46:03.582851 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:46:03.582851 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application with Lock( Sep 01 14:46:03.582851 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:46:03.582851 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:46:03.582851 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:46:03.582851 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:46:03.582851 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:46:03.582851 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:46:03.584337 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:46:03.584337 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:46:03.584337 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 14:46:03.584337 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 14:46:03.584337 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 14:46:03.584337 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 14:46:03.584337 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 14:46:03.584337 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 14:46:03.584337 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 01 14:46:03.584337 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Sep 01 14:46:03.584337 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 242/502] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Sep 1 14:46:03 2022] GET /identity/v3/users/glance => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:46:03.584337 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc1dca7-c367-47a1-ad29-346ef3a6643e None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:03.626597 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:03.631060 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 261/503] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Sep 1 14:46:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 14:46:03.631862 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69c5120-5ef6-4c10-a27c-8d114c59bb0c None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:03.694411 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc1dca7-c367-47a1-ad29-346ef3a6643e None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:03.728322 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805f036e-cffe-4689-be19-ddc02cbe79bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:03.728918 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805f036e-cffe-4689-be19-ddc02cbe79bc None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:03.729193 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805f036e-cffe-4689-be19-ddc02cbe79bc None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:03.729460 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805f036e-cffe-4689-be19-ddc02cbe79bc None admin] PATH_INFO: `/v3/roles` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:03.730115 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805f036e-cffe-4689-be19-ddc02cbe79bc None admin] RBAC: Adding query filter params (name=service) {{(pid=81341) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:46:03.730445 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805f036e-cffe-4689-be19-ddc02cbe79bc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:03.731818 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805f036e-cffe-4689-be19-ddc02cbe79bc None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:03.744385 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69c5120-5ef6-4c10-a27c-8d114c59bb0c None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:03.744544 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 243/504] 10.222.0.33 () {56 vars in 1103 bytes} [Thu Sep 1 14:46:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:46:03.744544 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805f036e-cffe-4689-be19-ddc02cbe79bc None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:03.774485 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595f68d7-f85d-4063-b613-fbb0ca304303 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:03.775104 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595f68d7-f85d-4063-b613-fbb0ca304303 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:03.775378 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595f68d7-f85d-4063-b613-fbb0ca304303 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:03.775654 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595f68d7-f85d-4063-b613-fbb0ca304303 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:03.777206 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:03.786039 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595f68d7-f85d-4063-b613-fbb0ca304303 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 14:46:03.790822 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application [None req-595f68d7-f85d-4063-b613-fbb0ca304303 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 14:46:03.790822 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:46:03.790822 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:46:03.790822 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:46:03.790822 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:46:03.790822 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:46:03.790822 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:46:03.790822 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:46:03.790822 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:46:03.790822 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:46:03.790822 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:46:03.790822 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:46:03.790822 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 14:46:03.790822 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 14:46:03.790822 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 14:46:03.790822 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 14:46:03.791845 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 14:46:03.791845 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 14:46:03.791845 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 14:46:03.791845 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 14:46:03.791845 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:46:03.791845 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:46:03.791845 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:46:03.791845 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:46:03.791845 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:46:03.791845 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:46:03.791845 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:46:03.791845 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application with Lock( Sep 01 14:46:03.791845 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:46:03.791845 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:46:03.791845 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:46:03.791845 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:46:03.792669 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:46:03.792669 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:46:03.792669 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:46:03.792669 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:46:03.792669 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 14:46:03.792669 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 14:46:03.792669 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 14:46:03.792669 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 14:46:03.792669 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 14:46:03.792669 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 14:46:03.792669 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 14:46:03.792669 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Sep 01 14:46:03.792669 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 262/505] 10.222.0.33 () {56 vars in 1102 bytes} [Thu Sep 1 14:46:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:46:03.792669 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595f68d7-f85d-4063-b613-fbb0ca304303 None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:03.841796 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805f036e-cffe-4689-be19-ddc02cbe79bc None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:03.850704 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:03.874853 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e03b31-0102-4a42-b9f2-f25b1d1b7d70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:03.875444 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e03b31-0102-4a42-b9f2-f25b1d1b7d70 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:03.875719 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e03b31-0102-4a42-b9f2-f25b1d1b7d70 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:03.875984 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e03b31-0102-4a42-b9f2-f25b1d1b7d70 None admin] PATH_INFO: `/v3/users` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:03.876698 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e03b31-0102-4a42-b9f2-f25b1d1b7d70 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81341) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:46:03.876994 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e03b31-0102-4a42-b9f2-f25b1d1b7d70 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:03.878427 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e03b31-0102-4a42-b9f2-f25b1d1b7d70 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:03.890791 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595f68d7-f85d-4063-b613-fbb0ca304303 None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:03.914036 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 244/506] 10.222.0.33 () {56 vars in 1137 bytes} [Thu Sep 1 14:46:03 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:46:03.914820 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e03b31-0102-4a42-b9f2-f25b1d1b7d70 None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:03.928771 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af741d9-657c-4201-9b6e-4227d4102a56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:03.929399 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af741d9-657c-4201-9b6e-4227d4102a56 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:03.929681 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af741d9-657c-4201-9b6e-4227d4102a56 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:03.929951 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af741d9-657c-4201-9b6e-4227d4102a56 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:03.939999 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af741d9-657c-4201-9b6e-4227d4102a56 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:03.941455 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af741d9-657c-4201-9b6e-4227d4102a56 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:03.956821 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application [None req-2af741d9-657c-4201-9b6e-4227d4102a56 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 14:46:03.956821 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:46:03.956821 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:46:03.956821 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:46:03.956821 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:46:03.956821 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:46:03.956821 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:46:03.956821 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:46:03.956821 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:46:03.956821 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:46:03.956821 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:46:03.956821 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:46:03.956821 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 14:46:03.956821 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 14:46:03.956821 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 14:46:03.956821 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 14:46:03.957873 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:46:03.957873 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:46:03.957873 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:46:03.957873 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:46:03.957873 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:46:03.957873 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:46:03.957873 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:46:03.957873 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application with Lock( Sep 01 14:46:03.957873 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:46:03.957873 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:46:03.957873 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:46:03.957873 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:46:03.957873 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:46:03.957873 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:46:03.957873 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:46:03.957873 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:46:03.957873 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 14:46:03.958790 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 14:46:03.958790 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 14:46:03.958790 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Sep 01 14:46:03.958790 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 263/507] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 1 14:46:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:46:03.959005 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af741d9-657c-4201-9b6e-4227d4102a56 None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:04.008164 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:04.014923 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e03b31-0102-4a42-b9f2-f25b1d1b7d70 None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:04.047634 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6092bc61-5303-4d78-8895-b4ee3837141a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:04.048247 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6092bc61-5303-4d78-8895-b4ee3837141a None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:04.048522 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6092bc61-5303-4d78-8895-b4ee3837141a None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:04.048789 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6092bc61-5303-4d78-8895-b4ee3837141a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:04.055761 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af741d9-657c-4201-9b6e-4227d4102a56 None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:04.059220 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6092bc61-5303-4d78-8895-b4ee3837141a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:04.060675 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6092bc61-5303-4d78-8895-b4ee3837141a None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:04.068930 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:04.076881 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application [None req-6092bc61-5303-4d78-8895-b4ee3837141a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 14:46:04.076881 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:46:04.076881 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:46:04.076881 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:46:04.076881 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:46:04.076881 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:46:04.076881 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:46:04.076881 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:46:04.076881 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:46:04.076881 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:46:04.076881 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:46:04.076881 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:46:04.076881 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 14:46:04.076881 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 14:46:04.076881 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 14:46:04.076881 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 14:46:04.077982 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:46:04.077982 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:46:04.077982 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:46:04.077982 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:46:04.077982 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:46:04.077982 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:46:04.077982 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:46:04.077982 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application with Lock( Sep 01 14:46:04.077982 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:46:04.077982 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:46:04.077982 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:46:04.077982 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:46:04.077982 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:46:04.077982 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:46:04.077982 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:46:04.077982 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:46:04.077982 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 14:46:04.079261 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 14:46:04.079261 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 14:46:04.079261 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Sep 01 14:46:04.079261 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 245/508] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 1 14:46:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:46:04.079261 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6092bc61-5303-4d78-8895-b4ee3837141a None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:04.093387 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1208060-4f7d-4ee2-8c67-6490f066ac2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:04.094007 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1208060-4f7d-4ee2-8c67-6490f066ac2d None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:04.094307 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1208060-4f7d-4ee2-8c67-6490f066ac2d None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:04.094585 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1208060-4f7d-4ee2-8c67-6490f066ac2d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:04.104908 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1208060-4f7d-4ee2-8c67-6490f066ac2d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:04.106415 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1208060-4f7d-4ee2-8c67-6490f066ac2d None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:04.121815 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application [None req-e1208060-4f7d-4ee2-8c67-6490f066ac2d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 14:46:04.121815 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:46:04.121815 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:46:04.121815 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:46:04.121815 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:46:04.121815 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:46:04.121815 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:46:04.121815 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:46:04.121815 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:46:04.121815 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:46:04.121815 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:46:04.121815 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:46:04.121815 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 14:46:04.121815 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 14:46:04.121815 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 14:46:04.121815 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 14:46:04.122937 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:46:04.122937 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:46:04.122937 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:46:04.122937 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:46:04.122937 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:46:04.122937 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:46:04.122937 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:46:04.122937 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application with Lock( Sep 01 14:46:04.122937 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:46:04.122937 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:46:04.122937 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:46:04.122937 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:46:04.122937 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:46:04.122937 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:46:04.122937 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:46:04.122937 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:46:04.122937 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 14:46:04.123978 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 14:46:04.123978 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 14:46:04.123978 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Sep 01 14:46:04.123978 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 264/509] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 1 14:46:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:46:04.123978 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1208060-4f7d-4ee2-8c67-6490f066ac2d None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:04.183437 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6092bc61-5303-4d78-8895-b4ee3837141a None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:04.217181 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c77a3c-e2ca-4c45-a7c3-64a1b5d45e8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:04.217777 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c77a3c-e2ca-4c45-a7c3-64a1b5d45e8e None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:04.218051 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c77a3c-e2ca-4c45-a7c3-64a1b5d45e8e None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:04.218351 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c77a3c-e2ca-4c45-a7c3-64a1b5d45e8e None admin] PATH_INFO: `/v3/roles` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:04.219076 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c77a3c-e2ca-4c45-a7c3-64a1b5d45e8e None admin] RBAC: Adding query filter params (name=service) {{(pid=81341) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:46:04.219372 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c77a3c-e2ca-4c45-a7c3-64a1b5d45e8e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:04.220309 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1208060-4f7d-4ee2-8c67-6490f066ac2d None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:04.221014 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c77a3c-e2ca-4c45-a7c3-64a1b5d45e8e None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:04.229408 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 246/510] 10.222.0.33 () {56 vars in 1103 bytes} [Thu Sep 1 14:46:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:46:04.230185 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c77a3c-e2ca-4c45-a7c3-64a1b5d45e8e None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:04.240916 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:04.256174 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58188d42-6018-471f-a0cd-7a34b36dc41d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:04.256838 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58188d42-6018-471f-a0cd-7a34b36dc41d None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:04.257119 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58188d42-6018-471f-a0cd-7a34b36dc41d None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:04.257388 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58188d42-6018-471f-a0cd-7a34b36dc41d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:04.287377 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:04.324148 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58188d42-6018-471f-a0cd-7a34b36dc41d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:04.325612 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58188d42-6018-471f-a0cd-7a34b36dc41d None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:04.328672 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c77a3c-e2ca-4c45-a7c3-64a1b5d45e8e None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:04.361638 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418a2801-6868-46e3-8096-ebfb9e29a382 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:04.362247 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418a2801-6868-46e3-8096-ebfb9e29a382 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:04.362540 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418a2801-6868-46e3-8096-ebfb9e29a382 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:04.362810 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418a2801-6868-46e3-8096-ebfb9e29a382 None admin] PATH_INFO: `/v3/domains` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:04.363437 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418a2801-6868-46e3-8096-ebfb9e29a382 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81341) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:46:04.363738 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418a2801-6868-46e3-8096-ebfb9e29a382 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:04.365647 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418a2801-6868-46e3-8096-ebfb9e29a382 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:04.378091 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 247/511] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Sep 1 14:46:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:46:04.378906 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418a2801-6868-46e3-8096-ebfb9e29a382 None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:04.421730 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 265/512] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Sep 1 14:46:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 14:46:04.422534 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58188d42-6018-471f-a0cd-7a34b36dc41d None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:04.448801 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:04.481785 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:04.482042 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418a2801-6868-46e3-8096-ebfb9e29a382 None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:04.516570 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d18602-5696-4cdd-b6ef-b491ff963ae8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:04.517165 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d18602-5696-4cdd-b6ef-b491ff963ae8 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:04.517384 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58188d42-6018-471f-a0cd-7a34b36dc41d None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:04.517482 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d18602-5696-4cdd-b6ef-b491ff963ae8 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:04.517701 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d18602-5696-4cdd-b6ef-b491ff963ae8 None admin] PATH_INFO: `/v3/domains` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:04.518364 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d18602-5696-4cdd-b6ef-b491ff963ae8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81341) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:46:04.518673 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d18602-5696-4cdd-b6ef-b491ff963ae8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:04.520056 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d18602-5696-4cdd-b6ef-b491ff963ae8 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:04.531632 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 248/513] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Sep 1 14:46:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:46:04.532452 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d18602-5696-4cdd-b6ef-b491ff963ae8 None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:04.556243 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbfda93-3fc4-467e-a921-f092b3052bbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:04.556850 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbfda93-3fc4-467e-a921-f092b3052bbe None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:04.557126 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbfda93-3fc4-467e-a921-f092b3052bbe None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:04.557400 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbfda93-3fc4-467e-a921-f092b3052bbe None admin] PATH_INFO: `/v3/domains` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:04.558037 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbfda93-3fc4-467e-a921-f092b3052bbe None admin] RBAC: Adding query filter params (name=Default) {{(pid=81342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:46:04.558369 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbfda93-3fc4-467e-a921-f092b3052bbe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:04.559736 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbfda93-3fc4-467e-a921-f092b3052bbe None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:04.570930 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 266/514] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Sep 1 14:46:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:46:04.571709 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbfda93-3fc4-467e-a921-f092b3052bbe None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:04.629641 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d18602-5696-4cdd-b6ef-b491ff963ae8 None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:04.655766 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:04.665461 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7824c33a-6a04-44ed-988d-d2d5301db197 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:04.666085 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7824c33a-6a04-44ed-988d-d2d5301db197 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:04.666390 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7824c33a-6a04-44ed-988d-d2d5301db197 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:04.666656 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7824c33a-6a04-44ed-988d-d2d5301db197 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:04.673467 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbfda93-3fc4-467e-a921-f092b3052bbe None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:04.678088 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7824c33a-6a04-44ed-988d-d2d5301db197 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:04.679604 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7824c33a-6a04-44ed-988d-d2d5301db197 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:04.694100 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application [None req-7824c33a-6a04-44ed-988d-d2d5301db197 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 14:46:04.694100 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:46:04.694100 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:46:04.694100 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:46:04.694100 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:46:04.694100 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:46:04.694100 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:46:04.694100 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:46:04.694100 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:46:04.694100 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:46:04.694100 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:46:04.694100 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:46:04.694100 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 14:46:04.694100 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 14:46:04.694100 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 14:46:04.694100 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 14:46:04.695063 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:46:04.695063 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:46:04.695063 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:46:04.695063 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:46:04.695063 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:46:04.695063 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:46:04.695063 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:46:04.695063 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application with Lock( Sep 01 14:46:04.695063 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:46:04.695063 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:46:04.695063 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:46:04.695063 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:46:04.695063 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:46:04.695063 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:46:04.695063 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:46:04.695063 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:46:04.695063 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 14:46:04.696190 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 14:46:04.696190 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 14:46:04.696190 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Sep 01 14:46:04.696190 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 249/515] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 1 14:46:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:46:04.696407 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7824c33a-6a04-44ed-988d-d2d5301db197 None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:04.701475 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:04.713793 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4da0cf2-1255-4ea9-8e13-16f4fd637b35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:04.714449 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4da0cf2-1255-4ea9-8e13-16f4fd637b35 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:04.714726 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4da0cf2-1255-4ea9-8e13-16f4fd637b35 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:04.715000 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4da0cf2-1255-4ea9-8e13-16f4fd637b35 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:04.721745 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4da0cf2-1255-4ea9-8e13-16f4fd637b35 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:04.723313 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4da0cf2-1255-4ea9-8e13-16f4fd637b35 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:04.735643 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application [None req-d4da0cf2-1255-4ea9-8e13-16f4fd637b35 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 14:46:04.735643 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:46:04.735643 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:46:04.735643 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:46:04.735643 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:46:04.735643 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:46:04.735643 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:46:04.735643 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:46:04.735643 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:46:04.735643 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:46:04.735643 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:46:04.735643 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:46:04.735643 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 14:46:04.735643 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 14:46:04.735643 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 14:46:04.736516 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 14:46:04.736516 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:46:04.736516 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:46:04.736516 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:46:04.736516 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:46:04.736516 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:46:04.736516 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:46:04.736516 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:46:04.736516 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application with Lock( Sep 01 14:46:04.736516 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:46:04.736516 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:46:04.736516 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:46:04.736516 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:46:04.736516 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:46:04.736516 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:46:04.736516 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:46:04.736516 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:46:04.737384 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 14:46:04.737384 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 14:46:04.737384 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 14:46:04.737384 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 14:46:04.737384 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 14:46:04.737384 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 14:46:04.737384 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 14:46:04.737384 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Sep 01 14:46:04.737384 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 267/516] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Sep 1 14:46:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:46:04.737864 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4da0cf2-1255-4ea9-8e13-16f4fd637b35 None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:04.795728 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7824c33a-6a04-44ed-988d-d2d5301db197 None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:04.828195 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a16f6a-f058-4854-af75-303207576b8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:04.828806 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a16f6a-f058-4854-af75-303207576b8c None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:04.829079 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a16f6a-f058-4854-af75-303207576b8c None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:04.829349 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a16f6a-f058-4854-af75-303207576b8c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:04.837895 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4da0cf2-1255-4ea9-8e13-16f4fd637b35 None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:04.840116 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a16f6a-f058-4854-af75-303207576b8c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:04.841591 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a16f6a-f058-4854-af75-303207576b8c None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:04.855401 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application [None req-a2a16f6a-f058-4854-af75-303207576b8c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 14:46:04.855401 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:46:04.855401 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:46:04.855401 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:46:04.855401 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:46:04.855401 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:46:04.855401 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:46:04.855401 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:46:04.855401 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:46:04.855401 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:46:04.855401 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:46:04.855401 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:46:04.855401 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 14:46:04.855401 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 14:46:04.855401 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 14:46:04.855401 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 14:46:04.856481 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:46:04.856481 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:46:04.856481 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:46:04.856481 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:46:04.856481 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:46:04.856481 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:46:04.856481 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:46:04.856481 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application with Lock( Sep 01 14:46:04.856481 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:46:04.856481 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:46:04.856481 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:46:04.856481 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:46:04.856481 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:46:04.856481 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:46:04.856481 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:46:04.856481 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:46:04.856481 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 14:46:04.857642 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 14:46:04.857642 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 14:46:04.857642 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Sep 01 14:46:04.857642 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 250/517] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 1 14:46:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:46:04.857642 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a16f6a-f058-4854-af75-303207576b8c None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:04.872646 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca121685-66bc-443d-ae7b-f65216abceff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:04.873242 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca121685-66bc-443d-ae7b-f65216abceff None admin] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:04.873514 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca121685-66bc-443d-ae7b-f65216abceff None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:04.873784 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca121685-66bc-443d-ae7b-f65216abceff None admin] PATH_INFO: `/v3/users` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:04.874446 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca121685-66bc-443d-ae7b-f65216abceff None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:04.875866 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca121685-66bc-443d-ae7b-f65216abceff None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:04.888427 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:04.897191 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 268/518] 10.222.0.33 () {60 vars in 1132 bytes} [Thu Sep 1 14:46:04 2022] POST /identity/v3/users => generated 247 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:46:04.898332 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca121685-66bc-443d-ae7b-f65216abceff None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:04.961591 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:04.974078 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a16f6a-f058-4854-af75-303207576b8c None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:05.012151 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a143079-dbca-421f-be15-31b13aa2c3e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:05.012750 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a143079-dbca-421f-be15-31b13aa2c3e1 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:05.013041 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a143079-dbca-421f-be15-31b13aa2c3e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:05.013298 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a143079-dbca-421f-be15-31b13aa2c3e1 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:05.022576 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a143079-dbca-421f-be15-31b13aa2c3e1 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:05.024078 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a143079-dbca-421f-be15-31b13aa2c3e1 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:05.028488 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca121685-66bc-443d-ae7b-f65216abceff None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:05.036216 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application [None req-8a143079-dbca-421f-be15-31b13aa2c3e1 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 01 14:46:05.036216 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:46:05.036216 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:46:05.036216 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:46:05.036216 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:46:05.036216 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:46:05.036216 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:46:05.036216 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:46:05.036216 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:46:05.036216 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:46:05.036216 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:46:05.036216 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:46:05.036216 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 14:46:05.036216 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 14:46:05.036216 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 14:46:05.036216 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 14:46:05.037206 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:46:05.037206 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:46:05.037206 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 14:46:05.037206 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 14:46:05.037206 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 14:46:05.037206 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 14:46:05.037206 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:46:05.037206 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:46:05.037206 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:46:05.037206 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:46:05.037206 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:46:05.037206 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application with Lock( Sep 01 14:46:05.037206 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:46:05.037206 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:46:05.037206 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:46:05.037206 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:46:05.037206 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:46:05.037206 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:46:05.038130 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:46:05.038130 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:46:05.038130 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 14:46:05.038130 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 14:46:05.038130 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 14:46:05.038130 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 14:46:05.038130 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 14:46:05.038130 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 14:46:05.038130 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 01 14:46:05.038130 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Sep 01 14:46:05.038130 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 251/519] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Sep 1 14:46:04 2022] GET /identity/v3/users/cinder => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:46:05.038130 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a143079-dbca-421f-be15-31b13aa2c3e1 None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:05.072022 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9312e5c-906f-48c7-b72d-63839bf0950a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:05.072634 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9312e5c-906f-48c7-b72d-63839bf0950a None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:05.072912 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9312e5c-906f-48c7-b72d-63839bf0950a None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:05.073186 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9312e5c-906f-48c7-b72d-63839bf0950a None admin] PATH_INFO: `/v3/domains` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:05.073888 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9312e5c-906f-48c7-b72d-63839bf0950a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:46:05.074196 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9312e5c-906f-48c7-b72d-63839bf0950a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:05.075588 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9312e5c-906f-48c7-b72d-63839bf0950a None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:05.092665 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 269/520] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Sep 1 14:46:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:46:05.093506 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9312e5c-906f-48c7-b72d-63839bf0950a None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:05.148487 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:05.158709 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a143079-dbca-421f-be15-31b13aa2c3e1 None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:05.195085 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf4b344-12e3-4b07-b086-59c6fe30c706 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:05.196056 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf4b344-12e3-4b07-b086-59c6fe30c706 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:05.196337 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf4b344-12e3-4b07-b086-59c6fe30c706 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:05.196612 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf4b344-12e3-4b07-b086-59c6fe30c706 None admin] PATH_INFO: `/v3/projects` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:05.197292 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf4b344-12e3-4b07-b086-59c6fe30c706 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81341) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:46:05.197594 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf4b344-12e3-4b07-b086-59c6fe30c706 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:05.198958 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:05.200006 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf4b344-12e3-4b07-b086-59c6fe30c706 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:05.202141 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9312e5c-906f-48c7-b72d-63839bf0950a None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:05.219336 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 252/521] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Sep 1 14:46:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:46:05.220000 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf4b344-12e3-4b07-b086-59c6fe30c706 None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:05.241055 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292d5aa8-75b3-42b5-8fdf-cf3ec4bddaf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:05.241654 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292d5aa8-75b3-42b5-8fdf-cf3ec4bddaf0 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:05.241931 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292d5aa8-75b3-42b5-8fdf-cf3ec4bddaf0 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:05.242195 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292d5aa8-75b3-42b5-8fdf-cf3ec4bddaf0 None admin] PATH_INFO: `/v3/domains` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:05.242866 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292d5aa8-75b3-42b5-8fdf-cf3ec4bddaf0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:46:05.243164 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292d5aa8-75b3-42b5-8fdf-cf3ec4bddaf0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:05.244523 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292d5aa8-75b3-42b5-8fdf-cf3ec4bddaf0 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:05.256420 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 270/522] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Sep 1 14:46:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:46:05.257221 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292d5aa8-75b3-42b5-8fdf-cf3ec4bddaf0 None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:05.335785 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf4b344-12e3-4b07-b086-59c6fe30c706 None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:05.370106 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43646ec8-6e3a-4dd6-9d66-b4e551720f38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:05.370515 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292d5aa8-75b3-42b5-8fdf-cf3ec4bddaf0 None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:05.370752 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43646ec8-6e3a-4dd6-9d66-b4e551720f38 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:05.371030 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43646ec8-6e3a-4dd6-9d66-b4e551720f38 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:05.371298 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43646ec8-6e3a-4dd6-9d66-b4e551720f38 None admin] PATH_INFO: `/v3/users` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:05.372028 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43646ec8-6e3a-4dd6-9d66-b4e551720f38 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81341) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:46:05.372326 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43646ec8-6e3a-4dd6-9d66-b4e551720f38 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:05.373770 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43646ec8-6e3a-4dd6-9d66-b4e551720f38 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:05.393951 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 253/523] 10.222.0.33 () {56 vars in 1137 bytes} [Thu Sep 1 14:46:05 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:46:05.395678 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43646ec8-6e3a-4dd6-9d66-b4e551720f38 None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:05.408981 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:05.410167 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35041c0b-f90b-40bc-b19f-7020948afc18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:05.411078 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35041c0b-f90b-40bc-b19f-7020948afc18 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:05.411476 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35041c0b-f90b-40bc-b19f-7020948afc18 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:05.411831 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35041c0b-f90b-40bc-b19f-7020948afc18 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:05.421109 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35041c0b-f90b-40bc-b19f-7020948afc18 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:05.422749 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35041c0b-f90b-40bc-b19f-7020948afc18 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:05.436501 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application [None req-35041c0b-f90b-40bc-b19f-7020948afc18 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 01 14:46:05.436501 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:46:05.436501 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:46:05.436501 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:46:05.436501 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:46:05.436501 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:46:05.436501 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:46:05.436501 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:46:05.436501 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:46:05.436501 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:46:05.436501 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:46:05.436501 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:46:05.436501 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 14:46:05.436501 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 14:46:05.436501 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 14:46:05.436501 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 14:46:05.437749 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:46:05.437749 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:46:05.437749 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 14:46:05.437749 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 14:46:05.437749 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 14:46:05.437749 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 14:46:05.437749 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:46:05.437749 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:46:05.437749 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:46:05.437749 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:46:05.437749 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:46:05.437749 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application with Lock( Sep 01 14:46:05.437749 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:46:05.437749 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:46:05.437749 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:46:05.437749 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:46:05.437749 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:46:05.437749 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:46:05.438951 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:46:05.438951 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:46:05.438951 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 14:46:05.438951 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 14:46:05.438951 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 14:46:05.438951 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 14:46:05.438951 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 14:46:05.438951 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 14:46:05.438951 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 01 14:46:05.438951 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Sep 01 14:46:05.445517 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 271/524] 10.222.0.33 () {56 vars in 1102 bytes} [Thu Sep 1 14:46:05 2022] GET /identity/v3/users/neutron => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:46:05.445517 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35041c0b-f90b-40bc-b19f-7020948afc18 None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:05.465236 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:05.506846 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43646ec8-6e3a-4dd6-9d66-b4e551720f38 None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:05.542865 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35041c0b-f90b-40bc-b19f-7020948afc18 None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:05.543349 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6b6dde-9e6d-4264-8329-d09471e4bc7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:05.543953 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6b6dde-9e6d-4264-8329-d09471e4bc7a None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:05.544425 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6b6dde-9e6d-4264-8329-d09471e4bc7a None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:05.544697 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6b6dde-9e6d-4264-8329-d09471e4bc7a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:05.545541 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6b6dde-9e6d-4264-8329-d09471e4bc7a None admin] RBAC: Adding query filter params (role.id=5e3f992e804046f381b3de8bf2a512e2, scope.project.id=e8c85b4bd17e4e89b56bf0f831b474d5, user.id=4858bb8621854ec8aba836e7d9050498) {{(pid=81341) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:46:05.545843 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6b6dde-9e6d-4264-8329-d09471e4bc7a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:05.547356 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6b6dde-9e6d-4264-8329-d09471e4bc7a None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:05.553862 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 254/525] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Sep 1 14:46:05 2022] GET /identity/v3/role_assignments?user.id=4858bb8621854ec8aba836e7d9050498&scope.project.id=e8c85b4bd17e4e89b56bf0f831b474d5&role.id=5e3f992e804046f381b3de8bf2a512e2 => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:46:05.554735 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6b6dde-9e6d-4264-8329-d09471e4bc7a None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:05.599823 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c6c28c-8e10-448d-815c-e2c64ccd0c7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:05.600530 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c6c28c-8e10-448d-815c-e2c64ccd0c7b None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:05.600764 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c6c28c-8e10-448d-815c-e2c64ccd0c7b None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:05.601037 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c6c28c-8e10-448d-815c-e2c64ccd0c7b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:05.609332 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c6c28c-8e10-448d-815c-e2c64ccd0c7b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:05.611168 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c6c28c-8e10-448d-815c-e2c64ccd0c7b None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:05.624628 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application [None req-63c6c28c-8e10-448d-815c-e2c64ccd0c7b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 14:46:05.624628 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:46:05.624628 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:46:05.624628 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:46:05.624628 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:46:05.624628 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:46:05.624628 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:46:05.624628 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:46:05.624628 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:46:05.624628 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:46:05.624628 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:46:05.624628 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:46:05.624628 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 14:46:05.624628 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 14:46:05.624628 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 14:46:05.625651 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 14:46:05.625651 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:46:05.625651 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:46:05.625651 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:46:05.625651 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:46:05.625651 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:46:05.625651 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:46:05.625651 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:46:05.625651 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application with Lock( Sep 01 14:46:05.625651 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:46:05.625651 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:46:05.625651 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:46:05.625651 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:46:05.625651 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:46:05.625651 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:46:05.625651 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:46:05.625651 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:46:05.626548 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 14:46:05.626548 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 14:46:05.626548 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 14:46:05.626548 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 14:46:05.626548 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 14:46:05.626548 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 14:46:05.626548 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 14:46:05.626548 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Sep 01 14:46:05.626548 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 272/526] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Sep 1 14:46:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:46:05.627185 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c6c28c-8e10-448d-815c-e2c64ccd0c7b None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:05.662842 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:05.699151 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6b6dde-9e6d-4264-8329-d09471e4bc7a None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:05.731245 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:05.740808 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f72b56e-03ff-43fe-9f03-dd2799320ae6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:05.741417 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f72b56e-03ff-43fe-9f03-dd2799320ae6 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:05.741691 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f72b56e-03ff-43fe-9f03-dd2799320ae6 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:05.741955 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f72b56e-03ff-43fe-9f03-dd2799320ae6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:05.754278 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c6c28c-8e10-448d-815c-e2c64ccd0c7b None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:05.789644 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25ab45b-218b-4c88-b60b-78717c8691b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:05.790604 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25ab45b-218b-4c88-b60b-78717c8691b1 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:05.791076 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25ab45b-218b-4c88-b60b-78717c8691b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:05.791535 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25ab45b-218b-4c88-b60b-78717c8691b1 None admin] PATH_INFO: `/v3/users` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:05.792537 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25ab45b-218b-4c88-b60b-78717c8691b1 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:46:05.793061 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25ab45b-218b-4c88-b60b-78717c8691b1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:05.794747 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25ab45b-218b-4c88-b60b-78717c8691b1 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:05.814151 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 273/527] 10.222.0.33 () {56 vars in 1139 bytes} [Thu Sep 1 14:46:05 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:46:05.814475 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f72b56e-03ff-43fe-9f03-dd2799320ae6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:05.815501 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25ab45b-218b-4c88-b60b-78717c8691b1 None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:05.815953 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f72b56e-03ff-43fe-9f03-dd2799320ae6 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:05.901286 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:05.919923 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25ab45b-218b-4c88-b60b-78717c8691b1 None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:05.935242 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 255/528] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Sep 1 14:46:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 14:46:05.936270 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f72b56e-03ff-43fe-9f03-dd2799320ae6 None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:05.957578 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56aa2eb-9091-481e-b840-1affd3868321 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:05.959119 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56aa2eb-9091-481e-b840-1affd3868321 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:05.959594 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56aa2eb-9091-481e-b840-1affd3868321 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:05.960054 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56aa2eb-9091-481e-b840-1affd3868321 None admin] PATH_INFO: `/v3/projects` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:05.960951 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56aa2eb-9091-481e-b840-1affd3868321 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:46:05.961446 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56aa2eb-9091-481e-b840-1affd3868321 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:05.963086 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56aa2eb-9091-481e-b840-1affd3868321 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:05.974628 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:05.982893 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 274/529] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Sep 1 14:46:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:46:05.984084 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56aa2eb-9091-481e-b840-1affd3868321 None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:06.047463 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f72b56e-03ff-43fe-9f03-dd2799320ae6 None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:06.083965 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5973e0e4-8b34-485e-8b85-4396f8f344aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:06.084796 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5973e0e4-8b34-485e-8b85-4396f8f344aa None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:06.085211 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5973e0e4-8b34-485e-8b85-4396f8f344aa None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:06.085733 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5973e0e4-8b34-485e-8b85-4396f8f344aa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:06.092369 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56aa2eb-9091-481e-b840-1affd3868321 None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:06.128596 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e35077-9c64-41ab-94c4-f49feff62632 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:06.129533 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e35077-9c64-41ab-94c4-f49feff62632 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:06.129898 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e35077-9c64-41ab-94c4-f49feff62632 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:06.130312 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e35077-9c64-41ab-94c4-f49feff62632 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:06.131451 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:06.142944 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e35077-9c64-41ab-94c4-f49feff62632 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:06.144737 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e35077-9c64-41ab-94c4-f49feff62632 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:06.158613 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5973e0e4-8b34-485e-8b85-4396f8f344aa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:06.160071 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5973e0e4-8b34-485e-8b85-4396f8f344aa None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:06.163937 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application [None req-52e35077-9c64-41ab-94c4-f49feff62632 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 14:46:06.163937 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:46:06.163937 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:46:06.163937 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:46:06.163937 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:46:06.163937 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:46:06.163937 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:46:06.163937 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:46:06.163937 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:46:06.163937 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:46:06.163937 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:46:06.163937 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:46:06.163937 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 14:46:06.163937 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 14:46:06.163937 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 14:46:06.163937 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 14:46:06.164832 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:46:06.164832 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:46:06.164832 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:46:06.164832 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:46:06.164832 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:46:06.164832 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:46:06.164832 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:46:06.164832 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application with Lock( Sep 01 14:46:06.164832 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:46:06.164832 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:46:06.164832 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:46:06.164832 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:46:06.164832 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:46:06.164832 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:46:06.164832 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:46:06.164832 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:46:06.164832 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 14:46:06.165708 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 14:46:06.165708 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 14:46:06.165708 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Sep 01 14:46:06.167759 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 275/530] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 1 14:46:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:46:06.169209 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e35077-9c64-41ab-94c4-f49feff62632 None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:06.235829 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:06.291020 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 256/531] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Sep 1 14:46:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 14:46:06.291782 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5973e0e4-8b34-485e-8b85-4396f8f344aa None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:06.302836 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e35077-9c64-41ab-94c4-f49feff62632 None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:06.341116 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df600802-a804-4617-88c1-2a62bd6bddb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:06.342046 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df600802-a804-4617-88c1-2a62bd6bddb9 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:06.342689 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df600802-a804-4617-88c1-2a62bd6bddb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:06.343251 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df600802-a804-4617-88c1-2a62bd6bddb9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:06.344160 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df600802-a804-4617-88c1-2a62bd6bddb9 None admin] RBAC: Adding query filter params (role.id=5e3f992e804046f381b3de8bf2a512e2, scope.project.id=e8c85b4bd17e4e89b56bf0f831b474d5, user.id=3069f72ce98049b99c9d98229a62bb51) {{(pid=81342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:46:06.344675 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df600802-a804-4617-88c1-2a62bd6bddb9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:06.346436 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df600802-a804-4617-88c1-2a62bd6bddb9 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:06.352986 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 276/532] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Sep 1 14:46:06 2022] GET /identity/v3/role_assignments?user.id=3069f72ce98049b99c9d98229a62bb51&scope.project.id=e8c85b4bd17e4e89b56bf0f831b474d5&role.id=5e3f992e804046f381b3de8bf2a512e2 => generated 259 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:46:06.354113 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df600802-a804-4617-88c1-2a62bd6bddb9 None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:06.368857 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:06.396895 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5973e0e4-8b34-485e-8b85-4396f8f344aa None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:06.452107 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc09acb-1ac8-407b-be72-39aff1305211 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:06.452729 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc09acb-1ac8-407b-be72-39aff1305211 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:06.453003 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc09acb-1ac8-407b-be72-39aff1305211 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:06.453279 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc09acb-1ac8-407b-be72-39aff1305211 None admin] PATH_INFO: `/v3/domains` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:06.453917 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc09acb-1ac8-407b-be72-39aff1305211 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81341) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:46:06.454216 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc09acb-1ac8-407b-be72-39aff1305211 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:06.455648 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc09acb-1ac8-407b-be72-39aff1305211 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:06.480200 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 257/533] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Sep 1 14:46:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:46:06.482054 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df600802-a804-4617-88c1-2a62bd6bddb9 None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:06.485948 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc09acb-1ac8-407b-be72-39aff1305211 None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:06.498625 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:06.523524 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1725756-2030-4c8b-87af-c4e69a8ea615 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:06.524589 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1725756-2030-4c8b-87af-c4e69a8ea615 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:06.525090 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1725756-2030-4c8b-87af-c4e69a8ea615 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:06.525617 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1725756-2030-4c8b-87af-c4e69a8ea615 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:06.538298 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1725756-2030-4c8b-87af-c4e69a8ea615 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:06.540257 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1725756-2030-4c8b-87af-c4e69a8ea615 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:06.561441 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application [None req-e1725756-2030-4c8b-87af-c4e69a8ea615 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 14:46:06.561441 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:46:06.561441 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:46:06.561441 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:46:06.561441 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:46:06.561441 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:46:06.561441 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:46:06.561441 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:46:06.561441 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:46:06.561441 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:46:06.561441 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:46:06.561441 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:46:06.561441 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 14:46:06.561441 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 14:46:06.561441 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 14:46:06.561441 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 14:46:06.562470 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:46:06.562470 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:46:06.562470 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:46:06.562470 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:46:06.562470 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:46:06.562470 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:46:06.562470 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:46:06.562470 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application with Lock( Sep 01 14:46:06.562470 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:46:06.562470 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:46:06.562470 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:46:06.562470 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:46:06.562470 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:46:06.562470 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:46:06.562470 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:46:06.562470 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:46:06.562470 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 14:46:06.563353 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 14:46:06.563353 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 14:46:06.563353 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Sep 01 14:46:06.568179 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 277/534] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 1 14:46:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:46:06.575994 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1725756-2030-4c8b-87af-c4e69a8ea615 None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:06.656499 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc09acb-1ac8-407b-be72-39aff1305211 None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:06.693916 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec66e0c6-9218-4f07-af08-875ffa763c53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:06.694594 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec66e0c6-9218-4f07-af08-875ffa763c53 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:06.694872 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec66e0c6-9218-4f07-af08-875ffa763c53 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:06.695142 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec66e0c6-9218-4f07-af08-875ffa763c53 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:06.697575 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:06.702297 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1725756-2030-4c8b-87af-c4e69a8ea615 None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:06.704090 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec66e0c6-9218-4f07-af08-875ffa763c53 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:06.705272 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec66e0c6-9218-4f07-af08-875ffa763c53 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:06.719125 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application [None req-ec66e0c6-9218-4f07-af08-875ffa763c53 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 14:46:06.719125 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:46:06.719125 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:46:06.719125 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:46:06.719125 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:46:06.719125 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:46:06.719125 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:46:06.719125 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:46:06.719125 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:46:06.719125 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:46:06.719125 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:46:06.719125 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:46:06.719125 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 14:46:06.719125 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 14:46:06.719125 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 14:46:06.720072 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 14:46:06.720072 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:46:06.720072 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:46:06.720072 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:46:06.720072 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:46:06.720072 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:46:06.720072 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:46:06.720072 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:46:06.720072 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application with Lock( Sep 01 14:46:06.720072 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:46:06.720072 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:46:06.720072 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:46:06.720072 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:46:06.720072 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:46:06.720072 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:46:06.720072 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:46:06.720072 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:46:06.720942 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 14:46:06.720942 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 14:46:06.720942 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 14:46:06.720942 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 14:46:06.720942 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 14:46:06.720942 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 14:46:06.720942 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 14:46:06.720942 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Sep 01 14:46:06.720942 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 258/535] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Sep 1 14:46:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 14:46:06.728680 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec66e0c6-9218-4f07-af08-875ffa763c53 None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:06.744660 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdad700f-4000-49f2-a2b0-7eb711be452c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:06.745651 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdad700f-4000-49f2-a2b0-7eb711be452c None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:06.746131 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdad700f-4000-49f2-a2b0-7eb711be452c None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:06.746627 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdad700f-4000-49f2-a2b0-7eb711be452c None admin] PATH_INFO: `/v3/domains` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:06.747469 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdad700f-4000-49f2-a2b0-7eb711be452c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:46:06.747968 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdad700f-4000-49f2-a2b0-7eb711be452c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:06.749624 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdad700f-4000-49f2-a2b0-7eb711be452c None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:06.763880 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 278/536] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Sep 1 14:46:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:46:06.769545 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdad700f-4000-49f2-a2b0-7eb711be452c None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:06.814705 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:06.882892 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdad700f-4000-49f2-a2b0-7eb711be452c None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:06.920797 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271aa968-c96a-4ac6-bbaa-ec5a220cda8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:06.921822 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271aa968-c96a-4ac6-bbaa-ec5a220cda8c None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:06.922356 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271aa968-c96a-4ac6-bbaa-ec5a220cda8c None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:06.922830 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271aa968-c96a-4ac6-bbaa-ec5a220cda8c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:06.931451 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271aa968-c96a-4ac6-bbaa-ec5a220cda8c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:06.933201 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271aa968-c96a-4ac6-bbaa-ec5a220cda8c None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:06.949666 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application [None req-271aa968-c96a-4ac6-bbaa-ec5a220cda8c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 14:46:06.949666 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:46:06.949666 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:46:06.949666 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:46:06.949666 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:46:06.949666 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:46:06.949666 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:46:06.949666 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:46:06.949666 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:46:06.949666 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:46:06.949666 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:46:06.949666 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:46:06.949666 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 14:46:06.949666 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 14:46:06.949666 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 14:46:06.950606 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 14:46:06.950606 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:46:06.950606 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:46:06.950606 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:46:06.950606 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:46:06.950606 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:46:06.950606 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:46:06.950606 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:46:06.950606 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application with Lock( Sep 01 14:46:06.950606 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:46:06.950606 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:46:06.950606 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:46:06.950606 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:46:06.950606 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:46:06.950606 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:46:06.950606 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:46:06.950606 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:46:06.951590 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 14:46:06.951590 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 14:46:06.951590 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 14:46:06.951590 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 14:46:06.951590 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 14:46:06.951590 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 14:46:06.951590 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 14:46:06.951590 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Sep 01 14:46:06.951590 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 279/537] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Sep 1 14:46:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 14:46:06.952674 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acd626a-fa50-456e-9d9d-af6e4db3aaa3 None None] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:06.952674 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acd626a-fa50-456e-9d9d-af6e4db3aaa3 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:06.952912 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acd626a-fa50-456e-9d9d-af6e4db3aaa3 None None] PATH_INFO: `/` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:06.958388 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 280/538] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 1 14:46:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:46:06.959574 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3e6f38-b6e4-47a3-90cf-56da529ddf63 None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:06.959864 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3e6f38-b6e4-47a3-90cf-56da529ddf63 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:06.960126 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3e6f38-b6e4-47a3-90cf-56da529ddf63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:06.969967 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec66e0c6-9218-4f07-af08-875ffa763c53 None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:06.978096 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-7f3e6f38-b6e4-47a3-90cf-56da529ddf63 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:46:06.979412 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-7f3e6f38-b6e4-47a3-90cf-56da529ddf63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:46:07.017217 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9456bea2-bb55-4487-a7ea-c3b55323a797 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:07.017874 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9456bea2-bb55-4487-a7ea-c3b55323a797 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:07.018158 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9456bea2-bb55-4487-a7ea-c3b55323a797 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:07.018454 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9456bea2-bb55-4487-a7ea-c3b55323a797 None admin] PATH_INFO: `/v3/projects` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:07.019197 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9456bea2-bb55-4487-a7ea-c3b55323a797 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81341) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:46:07.019504 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9456bea2-bb55-4487-a7ea-c3b55323a797 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:07.022214 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9456bea2-bb55-4487-a7ea-c3b55323a797 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:07.043554 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 259/539] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Sep 1 14:46:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:46:07.044676 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9456bea2-bb55-4487-a7ea-c3b55323a797 None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:07.066616 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 281/540] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 1 14:46:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 14:46:07.068112 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3e6f38-b6e4-47a3-90cf-56da529ddf63 None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:07.104972 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:07.121495 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:07.169733 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9456bea2-bb55-4487-a7ea-c3b55323a797 None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:07.208016 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589898d8-47ea-4fd4-9759-03de05f875cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:07.208713 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589898d8-47ea-4fd4-9759-03de05f875cb None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:07.209096 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589898d8-47ea-4fd4-9759-03de05f875cb None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:07.209455 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589898d8-47ea-4fd4-9759-03de05f875cb None admin] PATH_INFO: `/v3/projects` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:07.210214 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589898d8-47ea-4fd4-9759-03de05f875cb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81341) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:46:07.210649 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589898d8-47ea-4fd4-9759-03de05f875cb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:07.212566 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589898d8-47ea-4fd4-9759-03de05f875cb None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:07.225949 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 260/541] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Sep 1 14:46:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:46:07.233632 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589898d8-47ea-4fd4-9759-03de05f875cb None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:07.332040 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3e6f38-b6e4-47a3-90cf-56da529ddf63 None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:07.373862 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589898d8-47ea-4fd4-9759-03de05f875cb None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:07.397203 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:07.397812 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d378141-ea89-469b-a160-d1bec990c84c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:07.398638 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d378141-ea89-469b-a160-d1bec990c84c None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:07.398917 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d378141-ea89-469b-a160-d1bec990c84c None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:07.399184 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d378141-ea89-469b-a160-d1bec990c84c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:07.399922 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d378141-ea89-469b-a160-d1bec990c84c None admin] RBAC: Adding query filter params (role.id=5e3f992e804046f381b3de8bf2a512e2, scope.project.id=e8c85b4bd17e4e89b56bf0f831b474d5, user.id=d3b81739e34e4e648f88b6d73fcb5e46) {{(pid=81342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:46:07.400224 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d378141-ea89-469b-a160-d1bec990c84c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:07.401656 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d378141-ea89-469b-a160-d1bec990c84c None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:07.415502 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 282/542] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Sep 1 14:46:07 2022] GET /identity/v3/role_assignments?user.id=d3b81739e34e4e648f88b6d73fcb5e46&scope.project.id=e8c85b4bd17e4e89b56bf0f831b474d5&role.id=5e3f992e804046f381b3de8bf2a512e2 => generated 259 bytes in 348 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:46:07.417425 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d378141-ea89-469b-a160-d1bec990c84c None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:07.443098 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520f7db5-1d0f-4429-bfd1-92745192ea9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:07.443729 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520f7db5-1d0f-4429-bfd1-92745192ea9b None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:07.444006 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520f7db5-1d0f-4429-bfd1-92745192ea9b None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:07.444339 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520f7db5-1d0f-4429-bfd1-92745192ea9b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:07.445084 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520f7db5-1d0f-4429-bfd1-92745192ea9b None admin] RBAC: Adding query filter params (role.id=5e3f992e804046f381b3de8bf2a512e2, scope.project.id=e8c85b4bd17e4e89b56bf0f831b474d5, user.id=2790149639aa418bbc4e4982e93eda44) {{(pid=81341) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:46:07.445907 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520f7db5-1d0f-4429-bfd1-92745192ea9b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:07.447460 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520f7db5-1d0f-4429-bfd1-92745192ea9b None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:07.464421 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 261/543] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Sep 1 14:46:07 2022] GET /identity/v3/role_assignments?user.id=2790149639aa418bbc4e4982e93eda44&scope.project.id=e8c85b4bd17e4e89b56bf0f831b474d5&role.id=5e3f992e804046f381b3de8bf2a512e2 => generated 259 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:46:07.547492 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:07.576220 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d378141-ea89-469b-a160-d1bec990c84c None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:07.624566 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8695599b-cee9-49eb-92b2-82cc0790a2b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:07.625139 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8695599b-cee9-49eb-92b2-82cc0790a2b7 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:07.625422 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8695599b-cee9-49eb-92b2-82cc0790a2b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:07.625689 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8695599b-cee9-49eb-92b2-82cc0790a2b7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:07.644665 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8695599b-cee9-49eb-92b2-82cc0790a2b7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 14:46:07.654896 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application [None req-8695599b-cee9-49eb-92b2-82cc0790a2b7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 14:46:07.654896 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:46:07.654896 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:46:07.654896 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:46:07.654896 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:46:07.654896 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:46:07.654896 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:46:07.654896 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:46:07.654896 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:46:07.654896 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:46:07.654896 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:46:07.654896 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:46:07.654896 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 14:46:07.654896 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 14:46:07.654896 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 14:46:07.654896 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 14:46:07.655859 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 14:46:07.655859 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 14:46:07.655859 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 14:46:07.655859 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 14:46:07.655859 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:46:07.655859 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:46:07.655859 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:46:07.655859 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:46:07.655859 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:46:07.655859 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:46:07.655859 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:46:07.655859 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application with Lock( Sep 01 14:46:07.655859 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:46:07.655859 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:46:07.655859 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:46:07.655859 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:46:07.656682 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:46:07.656682 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:46:07.656682 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:46:07.656682 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:46:07.656682 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 14:46:07.656682 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 14:46:07.656682 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 14:46:07.656682 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 14:46:07.656682 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 14:46:07.656682 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 14:46:07.656682 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 14:46:07.656682 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Sep 01 14:46:07.656682 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 283/544] 10.222.0.33 () {56 vars in 1102 bytes} [Thu Sep 1 14:46:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:46:07.664851 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520f7db5-1d0f-4429-bfd1-92745192ea9b None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:07.785562 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f35fea7-db89-4960-9a8f-1b1e2e95ec12 None None] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:07.786130 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f35fea7-db89-4960-9a8f-1b1e2e95ec12 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:07.786777 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f35fea7-db89-4960-9a8f-1b1e2e95ec12 None None] PATH_INFO: `/` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:07.788090 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 284/545] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 1 14:46:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:46:07.793439 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1352b911-a05d-43b2-b3e4-9a6614a7e22d None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:07.793722 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1352b911-a05d-43b2-b3e4-9a6614a7e22d None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:07.793996 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1352b911-a05d-43b2-b3e4-9a6614a7e22d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:07.814291 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-1352b911-a05d-43b2-b3e4-9a6614a7e22d None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:46:07.815483 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-1352b911-a05d-43b2-b3e4-9a6614a7e22d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:46:07.835712 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520f7db5-1d0f-4429-bfd1-92745192ea9b None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:07.898722 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f6a381-a7e5-4b89-9697-d84b24ccbec4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:07.899248 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f6a381-a7e5-4b89-9697-d84b24ccbec4 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:07.899517 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f6a381-a7e5-4b89-9697-d84b24ccbec4 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:07.899785 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f6a381-a7e5-4b89-9697-d84b24ccbec4 None admin] PATH_INFO: `/v3/roles` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:07.900445 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f6a381-a7e5-4b89-9697-d84b24ccbec4 None admin] RBAC: Adding query filter params (name=service) {{(pid=81341) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:46:07.900745 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f6a381-a7e5-4b89-9697-d84b24ccbec4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:07.903187 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f6a381-a7e5-4b89-9697-d84b24ccbec4 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:07.915568 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 262/546] 10.222.0.33 () {56 vars in 1103 bytes} [Thu Sep 1 14:46:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:46:07.921576 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f6a381-a7e5-4b89-9697-d84b24ccbec4 None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:07.927783 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 285/547] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 1 14:46:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 14:46:07.987661 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:08.063800 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f6a381-a7e5-4b89-9697-d84b24ccbec4 None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:08.116899 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d28db2-bacd-4c08-a1c4-101bfb225f11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:08.117545 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d28db2-bacd-4c08-a1c4-101bfb225f11 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:08.117825 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d28db2-bacd-4c08-a1c4-101bfb225f11 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:08.118098 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d28db2-bacd-4c08-a1c4-101bfb225f11 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:08.135105 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d28db2-bacd-4c08-a1c4-101bfb225f11 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:08.136572 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d28db2-bacd-4c08-a1c4-101bfb225f11 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:08.151879 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application [None req-92d28db2-bacd-4c08-a1c4-101bfb225f11 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 14:46:08.151879 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:46:08.151879 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:46:08.151879 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:46:08.151879 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:46:08.151879 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:46:08.151879 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:46:08.151879 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:46:08.151879 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:46:08.151879 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:46:08.151879 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:46:08.151879 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:46:08.151879 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 14:46:08.151879 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 14:46:08.151879 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 14:46:08.151879 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 14:46:08.152799 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:46:08.152799 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:46:08.152799 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:46:08.152799 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:46:08.152799 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:46:08.152799 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:46:08.152799 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:46:08.152799 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application with Lock( Sep 01 14:46:08.152799 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:46:08.152799 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:46:08.152799 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:46:08.152799 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:46:08.152799 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:46:08.152799 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:46:08.152799 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:46:08.152799 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:46:08.152799 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 14:46:08.153652 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 14:46:08.153652 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 14:46:08.153652 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Sep 01 14:46:08.153652 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 263/548] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 1 14:46:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:46:08.163310 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1352b911-a05d-43b2-b3e4-9a6614a7e22d None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:08.275481 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:08.294939 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d28db2-bacd-4c08-a1c4-101bfb225f11 None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:08.309847 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1352b911-a05d-43b2-b3e4-9a6614a7e22d None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:08.355980 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5479ac8-09a2-47d6-8d05-683954935892 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:08.357065 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5479ac8-09a2-47d6-8d05-683954935892 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:08.357486 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5479ac8-09a2-47d6-8d05-683954935892 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:08.358340 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5479ac8-09a2-47d6-8d05-683954935892 None admin] PATH_INFO: `/v3/domains` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:08.359413 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5479ac8-09a2-47d6-8d05-683954935892 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:46:08.359873 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5479ac8-09a2-47d6-8d05-683954935892 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:08.362068 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5479ac8-09a2-47d6-8d05-683954935892 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:08.378042 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 286/549] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Sep 1 14:46:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:46:08.387247 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5479ac8-09a2-47d6-8d05-683954935892 None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:08.400155 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:08.410017 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d28db2-bacd-4c08-a1c4-101bfb225f11 None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:08.458941 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335a3619-9ec5-46b7-b252-58c4b9c7d0fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:08.459943 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335a3619-9ec5-46b7-b252-58c4b9c7d0fa None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:08.460420 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335a3619-9ec5-46b7-b252-58c4b9c7d0fa None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:08.460883 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335a3619-9ec5-46b7-b252-58c4b9c7d0fa None admin] PATH_INFO: `/v3/roles/service` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:08.478170 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335a3619-9ec5-46b7-b252-58c4b9c7d0fa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 14:46:08.490009 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application [None req-335a3619-9ec5-46b7-b252-58c4b9c7d0fa None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 14:46:08.490009 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:46:08.490009 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:46:08.490009 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:46:08.490009 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:46:08.490009 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:46:08.490009 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:46:08.490009 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:46:08.490009 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:46:08.490009 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:46:08.490009 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:46:08.490009 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:46:08.490009 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 14:46:08.490009 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 14:46:08.490009 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 14:46:08.490009 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 14:46:08.491522 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 14:46:08.491522 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 14:46:08.491522 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 14:46:08.491522 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 14:46:08.491522 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:46:08.491522 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:46:08.491522 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:46:08.491522 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:46:08.491522 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:46:08.491522 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:46:08.491522 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:46:08.491522 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application with Lock( Sep 01 14:46:08.491522 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:46:08.491522 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:46:08.491522 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:46:08.491522 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:46:08.492526 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:46:08.492526 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:46:08.492526 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:46:08.492526 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:46:08.492526 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 14:46:08.492526 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 14:46:08.492526 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 14:46:08.492526 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 14:46:08.492526 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 14:46:08.492526 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 14:46:08.492526 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 14:46:08.492526 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Sep 01 14:46:08.494851 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 264/550] 10.222.0.33 () {56 vars in 1102 bytes} [Thu Sep 1 14:46:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 14:46:08.499844 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5479ac8-09a2-47d6-8d05-683954935892 None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:08.507362 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335a3619-9ec5-46b7-b252-58c4b9c7d0fa None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:08.528915 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:08.535138 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d483b09-84dd-4709-9f26-8ec6eac42186 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:08.535763 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d483b09-84dd-4709-9f26-8ec6eac42186 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:08.536061 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d483b09-84dd-4709-9f26-8ec6eac42186 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:08.536409 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d483b09-84dd-4709-9f26-8ec6eac42186 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:08.544182 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d483b09-84dd-4709-9f26-8ec6eac42186 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:08.546952 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d483b09-84dd-4709-9f26-8ec6eac42186 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:08.560241 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application [None req-2d483b09-84dd-4709-9f26-8ec6eac42186 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 01 14:46:08.560241 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:46:08.560241 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:46:08.560241 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:46:08.560241 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:46:08.560241 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:46:08.560241 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:46:08.560241 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:46:08.560241 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:46:08.560241 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:46:08.560241 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:46:08.560241 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:46:08.560241 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 14:46:08.560241 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 14:46:08.560241 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 14:46:08.560241 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 14:46:08.561195 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:46:08.561195 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:46:08.561195 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 14:46:08.561195 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 14:46:08.561195 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 14:46:08.561195 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 14:46:08.561195 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:46:08.561195 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:46:08.561195 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:46:08.561195 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:46:08.561195 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:46:08.561195 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application with Lock( Sep 01 14:46:08.561195 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:46:08.561195 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:46:08.561195 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:46:08.561195 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:46:08.561195 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:46:08.561195 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:46:08.562909 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:46:08.562909 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:46:08.562909 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 14:46:08.562909 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 14:46:08.562909 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 14:46:08.562909 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 14:46:08.562909 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 14:46:08.562909 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 14:46:08.562909 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 01 14:46:08.562909 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Sep 01 14:46:08.562909 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 287/551] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Sep 1 14:46:08 2022] GET /identity/v3/users/swift => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:46:08.566810 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d483b09-84dd-4709-9f26-8ec6eac42186 None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:08.628388 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335a3619-9ec5-46b7-b252-58c4b9c7d0fa None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:08.710978 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba8c4df-5653-4671-846e-18f94390d986 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:08.711638 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba8c4df-5653-4671-846e-18f94390d986 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:08.711871 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba8c4df-5653-4671-846e-18f94390d986 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:08.712150 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba8c4df-5653-4671-846e-18f94390d986 None admin] PATH_INFO: `/v3/roles` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:08.712832 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba8c4df-5653-4671-846e-18f94390d986 None admin] RBAC: Adding query filter params (name=service) {{(pid=81341) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:46:08.713137 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba8c4df-5653-4671-846e-18f94390d986 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:08.714574 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba8c4df-5653-4671-846e-18f94390d986 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:08.717874 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d483b09-84dd-4709-9f26-8ec6eac42186 None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:08.744573 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 265/552] 10.222.0.33 () {56 vars in 1103 bytes} [Thu Sep 1 14:46:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:46:08.755334 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9d480a-a8af-44aa-a387-d47c7591ffdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:08.755592 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba8c4df-5653-4671-846e-18f94390d986 None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:08.756505 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9d480a-a8af-44aa-a387-d47c7591ffdf None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:08.756923 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9d480a-a8af-44aa-a387-d47c7591ffdf None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:08.757326 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9d480a-a8af-44aa-a387-d47c7591ffdf None admin] PATH_INFO: `/v3/users` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:08.758269 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9d480a-a8af-44aa-a387-d47c7591ffdf None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:46:08.758723 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9d480a-a8af-44aa-a387-d47c7591ffdf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:08.760332 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9d480a-a8af-44aa-a387-d47c7591ffdf None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:08.779594 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 288/553] 10.222.0.33 () {56 vars in 1135 bytes} [Thu Sep 1 14:46:08 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 14:46:08.780845 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815276ce-cf1c-4360-904f-a21a9bf8d96e None None] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:08.781125 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815276ce-cf1c-4360-904f-a21a9bf8d96e None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:08.781397 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815276ce-cf1c-4360-904f-a21a9bf8d96e None None] PATH_INFO: `/` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:08.781558 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:08.787333 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 289/554] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 1 14:46:08 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:46:08.789025 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815276ce-cf1c-4360-904f-a21a9bf8d96e None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:08.844573 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:08.882902 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba8c4df-5653-4671-846e-18f94390d986 None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:08.898545 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815276ce-cf1c-4360-904f-a21a9bf8d96e None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:08.920317 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9c7889-f95e-4c6c-9c7a-5ad2ec2c5430 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:08.920915 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9c7889-f95e-4c6c-9c7a-5ad2ec2c5430 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:08.921306 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9c7889-f95e-4c6c-9c7a-5ad2ec2c5430 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:08.921586 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9c7889-f95e-4c6c-9c7a-5ad2ec2c5430 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:08.935004 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9c7889-f95e-4c6c-9c7a-5ad2ec2c5430 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:08.937347 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82716b86-5384-4534-85d2-0c564d03d930 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:08.938055 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82716b86-5384-4534-85d2-0c564d03d930 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:08.938512 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82716b86-5384-4534-85d2-0c564d03d930 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:08.938908 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82716b86-5384-4534-85d2-0c564d03d930 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:08.940715 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9c7889-f95e-4c6c-9c7a-5ad2ec2c5430 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:08.956398 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application [None req-0d9c7889-f95e-4c6c-9c7a-5ad2ec2c5430 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 14:46:08.956398 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:46:08.956398 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:46:08.956398 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:46:08.956398 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:46:08.956398 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:46:08.956398 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:46:08.956398 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:46:08.956398 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:46:08.956398 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:46:08.956398 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:46:08.956398 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:46:08.956398 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 14:46:08.956398 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 14:46:08.956398 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 14:46:08.956398 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 14:46:08.957425 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:46:08.957425 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:46:08.957425 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:46:08.957425 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:46:08.957425 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:46:08.957425 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:46:08.957425 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:46:08.957425 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application with Lock( Sep 01 14:46:08.957425 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:46:08.957425 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:46:08.957425 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:46:08.957425 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:46:08.957425 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:46:08.957425 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:46:08.957425 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:46:08.957425 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:46:08.957425 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 14:46:08.958882 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 14:46:08.958882 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 14:46:08.958882 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Sep 01 14:46:08.962711 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 266/555] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 1 14:46:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:46:08.964021 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace971b6-d56e-4f46-a9cc-44b14ef762a0 None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:08.964308 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace971b6-d56e-4f46-a9cc-44b14ef762a0 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:08.964582 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace971b6-d56e-4f46-a9cc-44b14ef762a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:08.983649 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-ace971b6-d56e-4f46-a9cc-44b14ef762a0 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:46:08.984954 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-ace971b6-d56e-4f46-a9cc-44b14ef762a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:46:09.013536 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82716b86-5384-4534-85d2-0c564d03d930 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:09.015201 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82716b86-5384-4534-85d2-0c564d03d930 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:09.058615 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:09.067927 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 267/556] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 1 14:46:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 14:46:09.068798 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace971b6-d56e-4f46-a9cc-44b14ef762a0 None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:09.074945 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:09.126493 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 290/557] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Sep 1 14:46:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 01 14:46:09.128010 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82716b86-5384-4534-85d2-0c564d03d930 None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:09.238346 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace971b6-d56e-4f46-a9cc-44b14ef762a0 None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:09.274263 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f10d4ff-58e8-47b6-8f59-66ebc6167946 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:09.274263 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f10d4ff-58e8-47b6-8f59-66ebc6167946 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:09.274263 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f10d4ff-58e8-47b6-8f59-66ebc6167946 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:09.274263 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f10d4ff-58e8-47b6-8f59-66ebc6167946 None admin] PATH_INFO: `/v3/domains` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:09.274710 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f10d4ff-58e8-47b6-8f59-66ebc6167946 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81341) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:46:09.274710 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f10d4ff-58e8-47b6-8f59-66ebc6167946 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:09.274710 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f10d4ff-58e8-47b6-8f59-66ebc6167946 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:09.288506 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82716b86-5384-4534-85d2-0c564d03d930 None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:09.292018 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 268/558] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Sep 1 14:46:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:46:09.293392 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f10d4ff-58e8-47b6-8f59-66ebc6167946 None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:09.355224 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3804eaa-2712-460c-9f9b-2eb461f418ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:09.357109 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3804eaa-2712-460c-9f9b-2eb461f418ca None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:09.357389 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3804eaa-2712-460c-9f9b-2eb461f418ca None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:09.357661 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3804eaa-2712-460c-9f9b-2eb461f418ca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:09.381236 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3804eaa-2712-460c-9f9b-2eb461f418ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:09.384126 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3804eaa-2712-460c-9f9b-2eb461f418ca None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:09.403204 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:09.415465 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application [None req-a3804eaa-2712-460c-9f9b-2eb461f418ca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 14:46:09.415465 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:46:09.415465 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:46:09.415465 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:46:09.415465 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:46:09.415465 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:46:09.415465 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:46:09.415465 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:46:09.415465 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:46:09.415465 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:46:09.415465 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:46:09.415465 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:46:09.415465 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 14:46:09.415465 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 14:46:09.415465 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 14:46:09.415465 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 14:46:09.416439 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:46:09.416439 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:46:09.416439 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:46:09.416439 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:46:09.416439 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:46:09.416439 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:46:09.416439 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:46:09.416439 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application with Lock( Sep 01 14:46:09.416439 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:46:09.416439 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:46:09.416439 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:46:09.416439 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:46:09.416439 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:46:09.416439 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:46:09.416439 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:46:09.416439 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:46:09.416439 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 14:46:09.417342 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 14:46:09.417342 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 14:46:09.417342 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Sep 01 14:46:09.417515 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:09.421143 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 291/559] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 1 14:46:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 14:46:09.422118 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3804eaa-2712-460c-9f9b-2eb461f418ca None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:09.474263 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f10d4ff-58e8-47b6-8f59-66ebc6167946 None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:09.511081 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de31df2b-84b7-407e-8fc1-ddb6705581f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:09.511634 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de31df2b-84b7-407e-8fc1-ddb6705581f5 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:09.511912 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de31df2b-84b7-407e-8fc1-ddb6705581f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:09.512182 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de31df2b-84b7-407e-8fc1-ddb6705581f5 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:09.520817 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de31df2b-84b7-407e-8fc1-ddb6705581f5 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:09.522424 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de31df2b-84b7-407e-8fc1-ddb6705581f5 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:09.534500 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application [None req-de31df2b-84b7-407e-8fc1-ddb6705581f5 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 01 14:46:09.534500 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:46:09.534500 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:46:09.534500 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:46:09.534500 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:46:09.534500 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:46:09.534500 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:46:09.534500 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:46:09.534500 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:46:09.534500 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:46:09.534500 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:46:09.534500 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:46:09.534500 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 14:46:09.534500 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 14:46:09.534500 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 14:46:09.534500 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 14:46:09.537212 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:46:09.537212 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:46:09.537212 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 14:46:09.537212 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 14:46:09.537212 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 14:46:09.537212 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 14:46:09.537212 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:46:09.537212 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:46:09.537212 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:46:09.537212 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:46:09.537212 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:46:09.537212 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application with Lock( Sep 01 14:46:09.537212 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:46:09.537212 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:46:09.537212 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:46:09.537212 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:46:09.537212 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:46:09.537212 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:46:09.539556 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:46:09.539556 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:46:09.539556 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 14:46:09.539556 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 14:46:09.539556 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 14:46:09.539556 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 14:46:09.539556 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 14:46:09.539556 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 14:46:09.539556 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 01 14:46:09.539556 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Sep 01 14:46:09.539556 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 269/560] 10.222.0.33 () {56 vars in 1093 bytes} [Thu Sep 1 14:46:09 2022] GET /identity/v3/users/nova => generated 82 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 14:46:09.539556 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de31df2b-84b7-407e-8fc1-ddb6705581f5 None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:09.550256 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3804eaa-2712-460c-9f9b-2eb461f418ca None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:09.580322 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f379e0-c6f6-48d3-b815-c95c84d59178 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:09.581135 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f379e0-c6f6-48d3-b815-c95c84d59178 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:09.581419 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f379e0-c6f6-48d3-b815-c95c84d59178 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:09.581692 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f379e0-c6f6-48d3-b815-c95c84d59178 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:09.593060 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f379e0-c6f6-48d3-b815-c95c84d59178 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 14:46:09.598264 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application [None req-03f379e0-c6f6-48d3-b815-c95c84d59178 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 14:46:09.598264 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:46:09.598264 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:46:09.598264 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:46:09.598264 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:46:09.598264 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:46:09.598264 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:46:09.598264 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:46:09.598264 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:46:09.598264 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:46:09.598264 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:46:09.598264 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:46:09.598264 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 14:46:09.598264 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 14:46:09.598264 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 14:46:09.598264 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 14:46:09.600860 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 14:46:09.600860 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 14:46:09.600860 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 14:46:09.600860 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 14:46:09.600860 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:46:09.600860 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:46:09.600860 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:46:09.600860 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:46:09.600860 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:46:09.600860 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:46:09.600860 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:46:09.600860 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application with Lock( Sep 01 14:46:09.600860 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:46:09.600860 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:46:09.600860 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:46:09.600860 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:46:09.602418 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:46:09.602418 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:46:09.602418 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:46:09.602418 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:46:09.602418 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 14:46:09.602418 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 14:46:09.602418 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 14:46:09.602418 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 14:46:09.602418 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 14:46:09.602418 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 14:46:09.602418 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 14:46:09.602418 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Sep 01 14:46:09.602418 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 292/561] 10.222.0.33 () {56 vars in 1102 bytes} [Thu Sep 1 14:46:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:46:09.602418 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f379e0-c6f6-48d3-b815-c95c84d59178 None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:09.646499 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de31df2b-84b7-407e-8fc1-ddb6705581f5 None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:09.665623 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:09.676261 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:09.685796 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ac41c3-cccb-4734-8743-1116e86d33f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:09.686432 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ac41c3-cccb-4734-8743-1116e86d33f0 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:09.686710 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ac41c3-cccb-4734-8743-1116e86d33f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:09.686984 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ac41c3-cccb-4734-8743-1116e86d33f0 None admin] PATH_INFO: `/v3/domains` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:09.687634 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ac41c3-cccb-4734-8743-1116e86d33f0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81341) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:46:09.687926 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ac41c3-cccb-4734-8743-1116e86d33f0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:09.689307 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ac41c3-cccb-4734-8743-1116e86d33f0 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:09.701830 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 270/562] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Sep 1 14:46:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:46:09.703156 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2c5949-9885-42f2-a32c-a86537a6b6ac None None] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:09.703441 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2c5949-9885-42f2-a32c-a86537a6b6ac None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:09.703714 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2c5949-9885-42f2-a32c-a86537a6b6ac None None] PATH_INFO: `/` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:09.705057 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 271/563] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 1 14:46:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:46:09.705643 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2c5949-9885-42f2-a32c-a86537a6b6ac None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:09.728790 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f379e0-c6f6-48d3-b815-c95c84d59178 None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:09.765439 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b491848-a128-476d-9a0f-37e33cd9e97d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:09.765965 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b491848-a128-476d-9a0f-37e33cd9e97d None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:09.766350 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b491848-a128-476d-9a0f-37e33cd9e97d None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:09.766627 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b491848-a128-476d-9a0f-37e33cd9e97d None admin] PATH_INFO: `/v3/users` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:09.767365 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b491848-a128-476d-9a0f-37e33cd9e97d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:46:09.767651 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b491848-a128-476d-9a0f-37e33cd9e97d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:09.769088 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b491848-a128-476d-9a0f-37e33cd9e97d None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:09.789124 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 293/564] 10.222.0.33 () {56 vars in 1133 bytes} [Thu Sep 1 14:46:09 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:46:09.789795 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b491848-a128-476d-9a0f-37e33cd9e97d None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:09.818355 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2c5949-9885-42f2-a32c-a86537a6b6ac None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:09.855468 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af3a291-a5fe-4129-a6b3-d8c777735bc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:09.856064 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af3a291-a5fe-4129-a6b3-d8c777735bc2 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:09.856345 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af3a291-a5fe-4129-a6b3-d8c777735bc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:09.856608 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af3a291-a5fe-4129-a6b3-d8c777735bc2 None admin] PATH_INFO: `/v3/roles` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:09.857269 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af3a291-a5fe-4129-a6b3-d8c777735bc2 None admin] RBAC: Adding query filter params (name=service) {{(pid=81341) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:46:09.857562 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af3a291-a5fe-4129-a6b3-d8c777735bc2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:09.858966 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af3a291-a5fe-4129-a6b3-d8c777735bc2 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:09.866777 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 272/565] 10.222.0.33 () {56 vars in 1103 bytes} [Thu Sep 1 14:46:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:46:09.867740 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60232aa1-b65f-43fa-8fdf-2fbc3ebb4e77 None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:09.868019 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60232aa1-b65f-43fa-8fdf-2fbc3ebb4e77 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:09.868288 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60232aa1-b65f-43fa-8fdf-2fbc3ebb4e77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:09.884842 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-60232aa1-b65f-43fa-8fdf-2fbc3ebb4e77 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:46:09.886026 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-60232aa1-b65f-43fa-8fdf-2fbc3ebb4e77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:46:09.893266 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b491848-a128-476d-9a0f-37e33cd9e97d None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:09.911826 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:09.915924 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:09.929936 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f56622c-bc16-4c7b-b003-cd58461abe24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:09.930598 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f56622c-bc16-4c7b-b003-cd58461abe24 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:09.930875 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f56622c-bc16-4c7b-b003-cd58461abe24 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:09.931137 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f56622c-bc16-4c7b-b003-cd58461abe24 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:09.937824 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f56622c-bc16-4c7b-b003-cd58461abe24 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:09.939407 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f56622c-bc16-4c7b-b003-cd58461abe24 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:09.950632 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application [None req-1f56622c-bc16-4c7b-b003-cd58461abe24 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 14:46:09.950632 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:46:09.950632 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:46:09.950632 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:46:09.950632 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:46:09.950632 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:46:09.950632 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:46:09.950632 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:46:09.950632 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:46:09.950632 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:46:09.950632 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:46:09.950632 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:46:09.950632 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 14:46:09.950632 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 14:46:09.950632 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 14:46:09.951581 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 14:46:09.951581 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:46:09.951581 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:46:09.951581 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:46:09.951581 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:46:09.951581 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:46:09.951581 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:46:09.951581 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:46:09.951581 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application with Lock( Sep 01 14:46:09.951581 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:46:09.951581 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:46:09.951581 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:46:09.951581 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:46:09.951581 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:46:09.951581 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:46:09.951581 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:46:09.951581 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:46:09.953138 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 14:46:09.953138 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 14:46:09.953138 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 14:46:09.953138 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 14:46:09.953138 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 14:46:09.953138 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 14:46:09.953138 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 14:46:09.953138 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Sep 01 14:46:09.953138 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 294/566] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Sep 1 14:46:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:46:09.953138 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f56622c-bc16-4c7b-b003-cd58461abe24 None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:09.959287 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 273/567] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 1 14:46:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 14:46:09.960522 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590abea2-5c95-419c-b7c9-a6765ddad0db None None] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:09.960802 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590abea2-5c95-419c-b7c9-a6765ddad0db None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:09.961065 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590abea2-5c95-419c-b7c9-a6765ddad0db None None] PATH_INFO: `/` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:09.962288 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 274/568] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 1 14:46:09 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:46:09.962963 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590abea2-5c95-419c-b7c9-a6765ddad0db None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:10.060097 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f56622c-bc16-4c7b-b003-cd58461abe24 None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:10.064811 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590abea2-5c95-419c-b7c9-a6765ddad0db None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:10.095756 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbb41eb-94da-41e2-8fdd-278060583724 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:10.096370 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbb41eb-94da-41e2-8fdd-278060583724 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:10.096644 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbb41eb-94da-41e2-8fdd-278060583724 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:10.096913 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbb41eb-94da-41e2-8fdd-278060583724 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:10.102197 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88705e08-341d-43db-aed0-89e83ee48fb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:10.102842 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88705e08-341d-43db-aed0-89e83ee48fb9 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:10.103120 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88705e08-341d-43db-aed0-89e83ee48fb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:10.103387 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88705e08-341d-43db-aed0-89e83ee48fb9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:10.107442 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbb41eb-94da-41e2-8fdd-278060583724 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:10.108887 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbb41eb-94da-41e2-8fdd-278060583724 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:10.113865 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88705e08-341d-43db-aed0-89e83ee48fb9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:10.115340 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88705e08-341d-43db-aed0-89e83ee48fb9 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:10.122808 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application [None req-1fbb41eb-94da-41e2-8fdd-278060583724 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 14:46:10.122808 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:46:10.122808 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:46:10.122808 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:46:10.122808 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:46:10.122808 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:46:10.122808 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:46:10.122808 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:46:10.122808 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:46:10.122808 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:46:10.122808 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:46:10.122808 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:46:10.122808 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 14:46:10.122808 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 14:46:10.122808 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 14:46:10.122808 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 14:46:10.123773 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:46:10.123773 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:46:10.123773 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:46:10.123773 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:46:10.123773 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:46:10.123773 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:46:10.123773 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:46:10.123773 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application with Lock( Sep 01 14:46:10.123773 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:46:10.123773 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:46:10.123773 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:46:10.123773 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:46:10.123773 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:46:10.123773 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:46:10.123773 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:46:10.123773 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:46:10.123773 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 14:46:10.124675 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 14:46:10.124675 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 14:46:10.124675 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Sep 01 14:46:10.124675 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 295/569] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 1 14:46:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:46:10.125784 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbb41eb-94da-41e2-8fdd-278060583724 None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:10.135066 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application [None req-88705e08-341d-43db-aed0-89e83ee48fb9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 14:46:10.135066 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:46:10.135066 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:46:10.135066 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:46:10.135066 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:46:10.135066 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:46:10.135066 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:46:10.135066 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:46:10.135066 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:46:10.135066 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:46:10.135066 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:46:10.135066 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:46:10.135066 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 14:46:10.135066 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 14:46:10.135066 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 14:46:10.135066 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 14:46:10.136021 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:46:10.136021 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:46:10.136021 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:46:10.136021 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:46:10.136021 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:46:10.136021 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:46:10.136021 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:46:10.136021 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application with Lock( Sep 01 14:46:10.136021 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:46:10.136021 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:46:10.136021 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:46:10.136021 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:46:10.136021 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:46:10.136021 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:46:10.136021 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:46:10.136021 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:46:10.136021 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 14:46:10.136921 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 14:46:10.136921 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 14:46:10.136921 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Sep 01 14:46:10.136921 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 275/570] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 1 14:46:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:46:10.137959 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2855ec6b-01ee-4527-9339-ac78f1cd3ccf None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:10.137959 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2855ec6b-01ee-4527-9339-ac78f1cd3ccf None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:10.137959 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2855ec6b-01ee-4527-9339-ac78f1cd3ccf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:10.157909 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-2855ec6b-01ee-4527-9339-ac78f1cd3ccf None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:46:10.159123 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-2855ec6b-01ee-4527-9339-ac78f1cd3ccf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:46:10.162652 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:10.173651 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:10.228413 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 276/571] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 1 14:46:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 14:46:10.229246 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2855ec6b-01ee-4527-9339-ac78f1cd3ccf None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:10.249445 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbb41eb-94da-41e2-8fdd-278060583724 None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:10.284680 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df9f69a-68ee-4e2c-b286-43c906f54eed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:10.285282 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df9f69a-68ee-4e2c-b286-43c906f54eed None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:10.285565 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df9f69a-68ee-4e2c-b286-43c906f54eed None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:10.285840 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df9f69a-68ee-4e2c-b286-43c906f54eed None admin] PATH_INFO: `/v3/projects` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:10.286598 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df9f69a-68ee-4e2c-b286-43c906f54eed None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:46:10.286903 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df9f69a-68ee-4e2c-b286-43c906f54eed None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:10.288343 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df9f69a-68ee-4e2c-b286-43c906f54eed None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:10.304732 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 296/572] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Sep 1 14:46:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:46:10.304732 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df9f69a-68ee-4e2c-b286-43c906f54eed None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:10.335521 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2855ec6b-01ee-4527-9339-ac78f1cd3ccf None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:10.370096 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a9e354-051c-4060-b7c7-954857bbc539 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:10.370724 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a9e354-051c-4060-b7c7-954857bbc539 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:10.370999 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a9e354-051c-4060-b7c7-954857bbc539 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:10.371266 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a9e354-051c-4060-b7c7-954857bbc539 None admin] PATH_INFO: `/v3/domains` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:10.371899 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a9e354-051c-4060-b7c7-954857bbc539 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81341) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:46:10.372194 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a9e354-051c-4060-b7c7-954857bbc539 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:10.373556 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a9e354-051c-4060-b7c7-954857bbc539 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:10.384933 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 277/573] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Sep 1 14:46:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:46:10.385743 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a9e354-051c-4060-b7c7-954857bbc539 None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:10.401341 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:10.404220 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:10.414742 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df9f69a-68ee-4e2c-b286-43c906f54eed None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:10.451819 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920f0759-b7ce-45ed-8996-7ad40b387b8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:10.452518 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920f0759-b7ce-45ed-8996-7ad40b387b8d None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:10.452795 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920f0759-b7ce-45ed-8996-7ad40b387b8d None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:10.453125 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920f0759-b7ce-45ed-8996-7ad40b387b8d None admin] PATH_INFO: `/v3/domains` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:10.453696 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920f0759-b7ce-45ed-8996-7ad40b387b8d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:46:10.453994 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920f0759-b7ce-45ed-8996-7ad40b387b8d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:10.455435 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920f0759-b7ce-45ed-8996-7ad40b387b8d None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:10.471450 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 297/574] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Sep 1 14:46:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:46:10.471450 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920f0759-b7ce-45ed-8996-7ad40b387b8d None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:10.499146 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a9e354-051c-4060-b7c7-954857bbc539 None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:10.532491 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cdaeb3-e9b6-4d8d-baa2-f1745656f14f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:10.533113 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cdaeb3-e9b6-4d8d-baa2-f1745656f14f None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:10.533386 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cdaeb3-e9b6-4d8d-baa2-f1745656f14f None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:10.533658 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cdaeb3-e9b6-4d8d-baa2-f1745656f14f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:10.543369 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cdaeb3-e9b6-4d8d-baa2-f1745656f14f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 14:46:10.548499 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application [None req-73cdaeb3-e9b6-4d8d-baa2-f1745656f14f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 14:46:10.548499 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:46:10.548499 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:46:10.548499 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:46:10.548499 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:46:10.548499 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:46:10.548499 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:46:10.548499 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:46:10.548499 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:46:10.548499 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:46:10.548499 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:46:10.548499 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:46:10.548499 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 14:46:10.548499 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 14:46:10.548499 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 14:46:10.548499 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 14:46:10.549463 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 14:46:10.549463 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 14:46:10.549463 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 14:46:10.549463 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 14:46:10.549463 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:46:10.549463 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:46:10.549463 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:46:10.549463 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:46:10.549463 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:46:10.549463 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:46:10.549463 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:46:10.549463 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application with Lock( Sep 01 14:46:10.549463 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:46:10.549463 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:46:10.549463 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:46:10.549463 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:46:10.550346 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:46:10.550346 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:46:10.550346 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:46:10.550346 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:46:10.550346 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 14:46:10.550346 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 14:46:10.550346 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 14:46:10.550346 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 14:46:10.550346 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 14:46:10.550346 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 14:46:10.550346 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 14:46:10.550346 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Sep 01 14:46:10.550346 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 278/575] 10.222.0.33 () {56 vars in 1102 bytes} [Thu Sep 1 14:46:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:46:10.550346 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cdaeb3-e9b6-4d8d-baa2-f1745656f14f None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:10.577229 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920f0759-b7ce-45ed-8996-7ad40b387b8d None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:10.611144 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63b3b8c-3c8b-4967-be63-68a0912e516a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:10.611680 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63b3b8c-3c8b-4967-be63-68a0912e516a None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:10.611957 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63b3b8c-3c8b-4967-be63-68a0912e516a None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:10.612224 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63b3b8c-3c8b-4967-be63-68a0912e516a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:10.612941 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63b3b8c-3c8b-4967-be63-68a0912e516a None admin] RBAC: Adding query filter params (role.id=5e3f992e804046f381b3de8bf2a512e2, scope.project.id=e8c85b4bd17e4e89b56bf0f831b474d5, user.id=73a5bda5a6ea46e9863ccafd57dee339) {{(pid=81342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:46:10.613248 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63b3b8c-3c8b-4967-be63-68a0912e516a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:10.615241 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63b3b8c-3c8b-4967-be63-68a0912e516a None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:10.617434 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:10.620806 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 298/576] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Sep 1 14:46:10 2022] GET /identity/v3/role_assignments?user.id=73a5bda5a6ea46e9863ccafd57dee339&scope.project.id=e8c85b4bd17e4e89b56bf0f831b474d5&role.id=5e3f992e804046f381b3de8bf2a512e2 => generated 259 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:46:10.621983 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63b3b8c-3c8b-4967-be63-68a0912e516a None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:10.640111 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:10.666607 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cdaeb3-e9b6-4d8d-baa2-f1745656f14f None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:10.708299 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37da842b-337a-4457-a816-80ea52dfdedd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:10.708949 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37da842b-337a-4457-a816-80ea52dfdedd None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:10.709229 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37da842b-337a-4457-a816-80ea52dfdedd None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:10.709500 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37da842b-337a-4457-a816-80ea52dfdedd None admin] PATH_INFO: `/v3/projects/service` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:10.716918 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37da842b-337a-4457-a816-80ea52dfdedd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:10.718438 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37da842b-337a-4457-a816-80ea52dfdedd None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:10.731111 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application [None req-37da842b-337a-4457-a816-80ea52dfdedd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 14:46:10.731111 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:46:10.731111 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:46:10.731111 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:46:10.731111 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:46:10.731111 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:46:10.731111 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:46:10.731111 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:46:10.731111 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:46:10.731111 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:46:10.731111 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:46:10.731111 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:46:10.731111 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 14:46:10.731111 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 14:46:10.731111 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 14:46:10.732127 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 14:46:10.732127 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:46:10.732127 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:46:10.732127 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:46:10.732127 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:46:10.732127 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:46:10.732127 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:46:10.732127 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:46:10.732127 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application with Lock( Sep 01 14:46:10.732127 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:46:10.732127 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:46:10.732127 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:46:10.732127 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:46:10.732127 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:46:10.732127 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:46:10.732127 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:46:10.732127 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:46:10.733057 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 14:46:10.733057 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 14:46:10.733057 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 14:46:10.733057 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 14:46:10.733057 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 14:46:10.733057 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 14:46:10.733057 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 14:46:10.733057 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Sep 01 14:46:10.733057 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 279/577] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Sep 1 14:46:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:46:10.733057 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37da842b-337a-4457-a816-80ea52dfdedd None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:10.749275 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63b3b8c-3c8b-4967-be63-68a0912e516a None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:10.794712 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1469a36f-747e-4dde-95cd-89ff58f3b7d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:10.795684 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1469a36f-747e-4dde-95cd-89ff58f3b7d6 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:10.796380 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1469a36f-747e-4dde-95cd-89ff58f3b7d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:10.796669 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1469a36f-747e-4dde-95cd-89ff58f3b7d6 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:10.807948 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1469a36f-747e-4dde-95cd-89ff58f3b7d6 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:10.809663 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1469a36f-747e-4dde-95cd-89ff58f3b7d6 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:10.825889 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application [None req-1469a36f-747e-4dde-95cd-89ff58f3b7d6 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 01 14:46:10.825889 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:46:10.825889 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:46:10.825889 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:46:10.825889 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:46:10.825889 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:46:10.825889 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:46:10.825889 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:46:10.825889 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:46:10.825889 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:46:10.825889 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:46:10.825889 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:46:10.825889 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 14:46:10.825889 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 14:46:10.825889 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 14:46:10.825889 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 14:46:10.826922 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:46:10.826922 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:46:10.826922 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 14:46:10.826922 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 14:46:10.826922 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 14:46:10.826922 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 14:46:10.826922 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:46:10.826922 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:46:10.826922 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:46:10.826922 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:46:10.826922 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:46:10.826922 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application with Lock( Sep 01 14:46:10.826922 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:46:10.826922 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:46:10.826922 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:46:10.826922 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:46:10.826922 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:46:10.826922 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:46:10.832782 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:46:10.832782 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:46:10.832782 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 14:46:10.832782 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 14:46:10.832782 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 14:46:10.832782 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 14:46:10.832782 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 14:46:10.832782 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 14:46:10.832782 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 01 14:46:10.832782 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Sep 01 14:46:10.832782 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 299/578] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Sep 1 14:46:10 2022] GET /identity/v3/users/glance => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:46:10.832782 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1469a36f-747e-4dde-95cd-89ff58f3b7d6 None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:10.844181 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37da842b-337a-4457-a816-80ea52dfdedd None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:10.878346 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:10.884889 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24062033-1860-4383-9779-65189fec5fbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:10.885573 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24062033-1860-4383-9779-65189fec5fbf None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:10.885934 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24062033-1860-4383-9779-65189fec5fbf None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:10.886294 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24062033-1860-4383-9779-65189fec5fbf None admin] PATH_INFO: `/v3/roles/service` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:10.897327 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24062033-1860-4383-9779-65189fec5fbf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 14:46:10.903488 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application [None req-24062033-1860-4383-9779-65189fec5fbf None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 14:46:10.903488 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:46:10.903488 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:46:10.903488 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:46:10.903488 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:46:10.903488 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:46:10.903488 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:46:10.903488 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:46:10.903488 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:46:10.903488 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:46:10.903488 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:46:10.903488 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:46:10.903488 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 14:46:10.903488 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 14:46:10.903488 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 14:46:10.903488 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 14:46:10.904503 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 14:46:10.904503 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 14:46:10.904503 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 14:46:10.904503 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 14:46:10.904503 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:46:10.904503 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:46:10.904503 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:46:10.904503 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:46:10.904503 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:46:10.904503 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:46:10.904503 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:46:10.904503 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application with Lock( Sep 01 14:46:10.904503 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:46:10.904503 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:46:10.904503 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:46:10.904503 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:46:10.905441 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:46:10.905441 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:46:10.905441 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:46:10.905441 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:46:10.905441 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 14:46:10.905441 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 14:46:10.905441 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 14:46:10.905441 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 14:46:10.905441 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 14:46:10.905441 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 14:46:10.905441 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 14:46:10.905441 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Sep 01 14:46:10.905441 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 280/579] 10.222.0.33 () {56 vars in 1102 bytes} [Thu Sep 1 14:46:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:46:10.905441 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24062033-1860-4383-9779-65189fec5fbf None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:10.918344 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:10.946615 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1469a36f-747e-4dde-95cd-89ff58f3b7d6 None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:10.981629 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c18054-367b-4192-b196-da22fe0dda6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:10.982256 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c18054-367b-4192-b196-da22fe0dda6b None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:10.982534 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c18054-367b-4192-b196-da22fe0dda6b None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:10.982828 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c18054-367b-4192-b196-da22fe0dda6b None admin] PATH_INFO: `/v3/roles` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:10.983485 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c18054-367b-4192-b196-da22fe0dda6b None admin] RBAC: Adding query filter params (name=service) {{(pid=81342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:46:10.983783 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c18054-367b-4192-b196-da22fe0dda6b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:10.985151 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c18054-367b-4192-b196-da22fe0dda6b None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:10.993685 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 300/580] 10.222.0.33 () {56 vars in 1103 bytes} [Thu Sep 1 14:46:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:46:10.994509 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c18054-367b-4192-b196-da22fe0dda6b None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:11.025867 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24062033-1860-4383-9779-65189fec5fbf None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:11.068828 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c10388f-4cec-42d1-903d-d81565b5b770 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:11.069721 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c10388f-4cec-42d1-903d-d81565b5b770 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:11.070182 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c10388f-4cec-42d1-903d-d81565b5b770 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:11.070889 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c10388f-4cec-42d1-903d-d81565b5b770 None admin] PATH_INFO: `/v3/projects` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:11.071724 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c10388f-4cec-42d1-903d-d81565b5b770 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81341) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:46:11.072174 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c10388f-4cec-42d1-903d-d81565b5b770 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:11.073836 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c10388f-4cec-42d1-903d-d81565b5b770 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:11.087853 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 281/581] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Sep 1 14:46:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:46:11.088729 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c10388f-4cec-42d1-903d-d81565b5b770 None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:11.102014 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c18054-367b-4192-b196-da22fe0dda6b None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:11.135605 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e511135d-4bb0-4277-9d07-59ed0b06893b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:11.136130 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e511135d-4bb0-4277-9d07-59ed0b06893b None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:11.136401 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e511135d-4bb0-4277-9d07-59ed0b06893b None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:11.136667 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e511135d-4bb0-4277-9d07-59ed0b06893b None admin] PATH_INFO: `/v3/users` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:11.137385 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e511135d-4bb0-4277-9d07-59ed0b06893b None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:46:11.137679 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e511135d-4bb0-4277-9d07-59ed0b06893b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:11.139120 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e511135d-4bb0-4277-9d07-59ed0b06893b None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:11.141563 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:11.150436 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:11.157522 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 301/582] 10.222.0.33 () {56 vars in 1137 bytes} [Thu Sep 1 14:46:10 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:46:11.158423 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e511135d-4bb0-4277-9d07-59ed0b06893b None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:11.208994 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c10388f-4cec-42d1-903d-d81565b5b770 None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:11.245779 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c398a0-26a9-4e98-846f-62b3630e01ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:11.246413 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c398a0-26a9-4e98-846f-62b3630e01ce None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:11.246688 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c398a0-26a9-4e98-846f-62b3630e01ce None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:11.246957 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c398a0-26a9-4e98-846f-62b3630e01ce None admin] PATH_INFO: `/v3/roles` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:11.247615 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c398a0-26a9-4e98-846f-62b3630e01ce None admin] RBAC: Adding query filter params (name=service) {{(pid=81341) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:46:11.247910 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c398a0-26a9-4e98-846f-62b3630e01ce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:11.249291 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c398a0-26a9-4e98-846f-62b3630e01ce None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:11.257918 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 282/583] 10.222.0.33 () {56 vars in 1103 bytes} [Thu Sep 1 14:46:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:46:11.258593 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c398a0-26a9-4e98-846f-62b3630e01ce None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:11.267380 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e511135d-4bb0-4277-9d07-59ed0b06893b None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:11.304814 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507e57b7-b3a6-4b65-a23b-e93c6d2dddff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:11.305435 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507e57b7-b3a6-4b65-a23b-e93c6d2dddff None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:11.305713 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507e57b7-b3a6-4b65-a23b-e93c6d2dddff None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:11.306177 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507e57b7-b3a6-4b65-a23b-e93c6d2dddff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:11.316533 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507e57b7-b3a6-4b65-a23b-e93c6d2dddff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:11.318015 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507e57b7-b3a6-4b65-a23b-e93c6d2dddff None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:11.333530 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application [None req-507e57b7-b3a6-4b65-a23b-e93c6d2dddff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 14:46:11.333530 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:46:11.333530 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:46:11.333530 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:46:11.333530 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:46:11.333530 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:46:11.333530 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:46:11.333530 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:46:11.333530 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:46:11.333530 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:46:11.333530 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:46:11.333530 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:46:11.333530 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 14:46:11.333530 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 14:46:11.333530 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 14:46:11.333530 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 14:46:11.334551 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:46:11.334551 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:46:11.334551 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:46:11.334551 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:46:11.334551 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:46:11.334551 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:46:11.334551 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:46:11.334551 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application with Lock( Sep 01 14:46:11.334551 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:46:11.334551 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:46:11.334551 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:46:11.334551 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:46:11.334551 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:46:11.334551 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:46:11.334551 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:46:11.334551 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:46:11.334551 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 14:46:11.335482 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 14:46:11.335482 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 14:46:11.335482 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Sep 01 14:46:11.335482 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 302/584] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 1 14:46:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:46:11.335994 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507e57b7-b3a6-4b65-a23b-e93c6d2dddff None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:11.376161 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c398a0-26a9-4e98-846f-62b3630e01ce None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:11.386704 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:11.389866 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:11.415196 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f372f5ec-5886-4270-a07b-44234bd60cb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:11.416090 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f372f5ec-5886-4270-a07b-44234bd60cb7 None admin] REQUEST_METHOD: `PUT` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:11.416553 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f372f5ec-5886-4270-a07b-44234bd60cb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:11.417030 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f372f5ec-5886-4270-a07b-44234bd60cb7 None admin] PATH_INFO: `/v3/projects/e8c85b4bd17e4e89b56bf0f831b474d5/users/4858bb8621854ec8aba836e7d9050498/roles/5e3f992e804046f381b3de8bf2a512e2` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:11.435708 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f372f5ec-5886-4270-a07b-44234bd60cb7 None admin] RBAC: Authorizing `identity:create_grant(project_id=e8c85b4bd17e4e89b56bf0f831b474d5, user_id=4858bb8621854ec8aba836e7d9050498, role_id=5e3f992e804046f381b3de8bf2a512e2)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:11.438103 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f372f5ec-5886-4270-a07b-44234bd60cb7 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:11.442089 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507e57b7-b3a6-4b65-a23b-e93c6d2dddff None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:11.453327 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 283/585] 10.222.0.33 () {58 vars in 1439 bytes} [Thu Sep 1 14:46:11 2022] PUT /identity/v3/projects/e8c85b4bd17e4e89b56bf0f831b474d5/users/4858bb8621854ec8aba836e7d9050498/roles/5e3f992e804046f381b3de8bf2a512e2 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:46:11.454433 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f372f5ec-5886-4270-a07b-44234bd60cb7 None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:11.499276 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71713f44-4df9-4ccf-a299-f9a6ef082f2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:11.500107 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71713f44-4df9-4ccf-a299-f9a6ef082f2e None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:11.500475 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71713f44-4df9-4ccf-a299-f9a6ef082f2e None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:11.500848 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71713f44-4df9-4ccf-a299-f9a6ef082f2e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:11.518212 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71713f44-4df9-4ccf-a299-f9a6ef082f2e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:11.520186 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71713f44-4df9-4ccf-a299-f9a6ef082f2e None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:11.542421 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application [None req-71713f44-4df9-4ccf-a299-f9a6ef082f2e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 14:46:11.542421 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:46:11.542421 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:46:11.542421 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:46:11.542421 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:46:11.542421 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:46:11.542421 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:46:11.542421 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:46:11.542421 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:46:11.542421 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:46:11.542421 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:46:11.542421 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:46:11.542421 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 14:46:11.542421 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 14:46:11.542421 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 14:46:11.542421 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 14:46:11.543630 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:46:11.543630 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:46:11.543630 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:46:11.543630 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:46:11.543630 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:46:11.543630 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:46:11.543630 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:46:11.543630 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application with Lock( Sep 01 14:46:11.543630 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:46:11.543630 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:46:11.543630 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:46:11.543630 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:46:11.543630 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:46:11.543630 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:46:11.543630 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:46:11.543630 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:46:11.543630 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 14:46:11.544593 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 14:46:11.544593 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 14:46:11.544593 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Sep 01 14:46:11.546763 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 303/586] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 1 14:46:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:46:11.548054 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71713f44-4df9-4ccf-a299-f9a6ef082f2e None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:11.594358 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f372f5ec-5886-4270-a07b-44234bd60cb7 None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:11.641940 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bba5702-0db9-449f-bcf0-ef6de1353031 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:11.642845 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bba5702-0db9-449f-bcf0-ef6de1353031 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:11.643127 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bba5702-0db9-449f-bcf0-ef6de1353031 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:11.644366 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bba5702-0db9-449f-bcf0-ef6de1353031 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:11.656589 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bba5702-0db9-449f-bcf0-ef6de1353031 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:11.658594 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bba5702-0db9-449f-bcf0-ef6de1353031 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:11.680280 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application [None req-6bba5702-0db9-449f-bcf0-ef6de1353031 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 14:46:11.680280 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:46:11.680280 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:46:11.680280 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:46:11.680280 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:46:11.680280 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:46:11.680280 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:46:11.680280 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:46:11.680280 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:46:11.680280 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:46:11.680280 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:46:11.680280 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:46:11.680280 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 14:46:11.680280 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 14:46:11.680280 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 14:46:11.680280 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 14:46:11.681274 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:46:11.681274 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:46:11.681274 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:46:11.681274 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:46:11.681274 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:46:11.681274 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:46:11.681274 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:46:11.681274 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application with Lock( Sep 01 14:46:11.681274 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:46:11.681274 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:46:11.681274 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:46:11.681274 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:46:11.681274 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:46:11.681274 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:46:11.681274 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:46:11.681274 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:46:11.681274 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 14:46:11.682192 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 14:46:11.682192 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 14:46:11.682192 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Sep 01 14:46:11.682810 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 284/587] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 1 14:46:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:46:11.682921 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:11.686265 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bba5702-0db9-449f-bcf0-ef6de1353031 None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:11.688266 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71713f44-4df9-4ccf-a299-f9a6ef082f2e None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:11.705948 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:11.728659 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f829fee3-c7ee-4684-9ea3-a5dd9ed6361b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:11.729270 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f829fee3-c7ee-4684-9ea3-a5dd9ed6361b None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:11.729551 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f829fee3-c7ee-4684-9ea3-a5dd9ed6361b None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:11.729822 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f829fee3-c7ee-4684-9ea3-a5dd9ed6361b None admin] PATH_INFO: `/v3/domains` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:11.730516 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f829fee3-c7ee-4684-9ea3-a5dd9ed6361b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:46:11.730822 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f829fee3-c7ee-4684-9ea3-a5dd9ed6361b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:11.732210 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f829fee3-c7ee-4684-9ea3-a5dd9ed6361b None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:11.749595 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 304/588] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Sep 1 14:46:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:46:11.750581 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f829fee3-c7ee-4684-9ea3-a5dd9ed6361b None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:11.818963 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bba5702-0db9-449f-bcf0-ef6de1353031 None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:11.858882 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6292d50-4bf2-479c-89da-7a9b598ef129 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:11.859641 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6292d50-4bf2-479c-89da-7a9b598ef129 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:11.859906 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6292d50-4bf2-479c-89da-7a9b598ef129 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:11.862270 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6292d50-4bf2-479c-89da-7a9b598ef129 None admin] PATH_INFO: `/v3/domains` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:11.862270 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6292d50-4bf2-479c-89da-7a9b598ef129 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81341) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:46:11.862270 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6292d50-4bf2-479c-89da-7a9b598ef129 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:11.862736 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6292d50-4bf2-479c-89da-7a9b598ef129 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:11.864386 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f829fee3-c7ee-4684-9ea3-a5dd9ed6361b None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:11.875954 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 285/589] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Sep 1 14:46:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:46:11.876907 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6292d50-4bf2-479c-89da-7a9b598ef129 None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:11.909384 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e0f62a-022e-438a-88dd-0d313eff6172 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:11.910051 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e0f62a-022e-438a-88dd-0d313eff6172 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:11.910492 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e0f62a-022e-438a-88dd-0d313eff6172 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:11.910765 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e0f62a-022e-438a-88dd-0d313eff6172 None admin] PATH_INFO: `/v3/domains` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:11.911700 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e0f62a-022e-438a-88dd-0d313eff6172 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:46:11.912003 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e0f62a-022e-438a-88dd-0d313eff6172 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:11.914063 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e0f62a-022e-438a-88dd-0d313eff6172 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:11.929400 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 305/590] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Sep 1 14:46:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:46:11.930688 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e0f62a-022e-438a-88dd-0d313eff6172 None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:11.955888 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:11.965582 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:11.998133 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6292d50-4bf2-479c-89da-7a9b598ef129 None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:12.039102 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a719603-52cf-467e-895e-f452e4d0d146 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:12.040072 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a719603-52cf-467e-895e-f452e4d0d146 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:12.040505 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a719603-52cf-467e-895e-f452e4d0d146 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:12.040920 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a719603-52cf-467e-895e-f452e4d0d146 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:12.048467 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a719603-52cf-467e-895e-f452e4d0d146 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:12.050175 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a719603-52cf-467e-895e-f452e4d0d146 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:12.061966 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e0f62a-022e-438a-88dd-0d313eff6172 None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:12.062879 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application [None req-6a719603-52cf-467e-895e-f452e4d0d146 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 01 14:46:12.062879 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:46:12.062879 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:46:12.062879 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:46:12.062879 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:46:12.062879 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:46:12.062879 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:46:12.062879 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:46:12.062879 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:46:12.062879 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:46:12.062879 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:46:12.062879 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:46:12.062879 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 14:46:12.062879 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 14:46:12.062879 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 14:46:12.062879 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 14:46:12.063997 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:46:12.063997 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:46:12.063997 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 14:46:12.063997 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 14:46:12.063997 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 14:46:12.063997 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 14:46:12.063997 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:46:12.063997 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:46:12.063997 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:46:12.063997 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:46:12.063997 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:46:12.063997 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application with Lock( Sep 01 14:46:12.063997 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:46:12.063997 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:46:12.063997 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:46:12.063997 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:46:12.063997 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:46:12.063997 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:46:12.065085 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:46:12.065085 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:46:12.065085 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 14:46:12.065085 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 14:46:12.065085 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 14:46:12.065085 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 14:46:12.065085 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 14:46:12.065085 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 14:46:12.065085 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 01 14:46:12.065085 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Sep 01 14:46:12.067150 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 286/591] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Sep 1 14:46:11 2022] GET /identity/v3/users/cinder => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:46:12.068234 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a719603-52cf-467e-895e-f452e4d0d146 None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:12.107852 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14f2e7c-f39c-463f-a7bb-9901a8ad9371 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:12.108494 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14f2e7c-f39c-463f-a7bb-9901a8ad9371 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:12.108772 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14f2e7c-f39c-463f-a7bb-9901a8ad9371 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:12.109039 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14f2e7c-f39c-463f-a7bb-9901a8ad9371 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:12.117933 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14f2e7c-f39c-463f-a7bb-9901a8ad9371 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:12.119658 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14f2e7c-f39c-463f-a7bb-9901a8ad9371 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:12.135482 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application [None req-a14f2e7c-f39c-463f-a7bb-9901a8ad9371 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 14:46:12.135482 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:46:12.135482 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:46:12.135482 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:46:12.135482 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:46:12.135482 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:46:12.135482 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:46:12.135482 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:46:12.135482 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:46:12.135482 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:46:12.135482 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:46:12.135482 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:46:12.135482 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 14:46:12.135482 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 14:46:12.135482 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 14:46:12.136417 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 14:46:12.136417 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:46:12.136417 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:46:12.136417 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:46:12.136417 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:46:12.136417 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:46:12.136417 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:46:12.136417 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:46:12.136417 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application with Lock( Sep 01 14:46:12.136417 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:46:12.136417 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:46:12.136417 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:46:12.136417 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:46:12.136417 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:46:12.136417 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:46:12.136417 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:46:12.136417 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:46:12.137372 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 14:46:12.137372 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 14:46:12.137372 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 14:46:12.137372 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 14:46:12.137372 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 14:46:12.137372 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 14:46:12.137372 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 14:46:12.137372 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Sep 01 14:46:12.137372 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 306/592] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Sep 1 14:46:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:46:12.138120 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14f2e7c-f39c-463f-a7bb-9901a8ad9371 None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:12.183069 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a719603-52cf-467e-895e-f452e4d0d146 None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:12.218529 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8461c857-d300-4eb6-8cad-3a0d9f17afa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:12.219513 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8461c857-d300-4eb6-8cad-3a0d9f17afa9 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:12.219938 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8461c857-d300-4eb6-8cad-3a0d9f17afa9 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:12.220342 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8461c857-d300-4eb6-8cad-3a0d9f17afa9 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:12.222555 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:12.230226 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8461c857-d300-4eb6-8cad-3a0d9f17afa9 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:12.232006 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8461c857-d300-4eb6-8cad-3a0d9f17afa9 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:12.244970 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application [None req-8461c857-d300-4eb6-8cad-3a0d9f17afa9 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 01 14:46:12.244970 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:46:12.244970 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:46:12.244970 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:46:12.244970 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:46:12.244970 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:46:12.244970 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:46:12.244970 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:46:12.244970 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:46:12.244970 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:46:12.244970 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:46:12.244970 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:46:12.244970 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 14:46:12.244970 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 14:46:12.244970 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 14:46:12.244970 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 14:46:12.246101 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:46:12.246101 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:46:12.246101 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 14:46:12.246101 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 14:46:12.246101 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 14:46:12.246101 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 14:46:12.246101 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:46:12.246101 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:46:12.246101 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:46:12.246101 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:46:12.246101 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:46:12.246101 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application with Lock( Sep 01 14:46:12.246101 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:46:12.246101 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:46:12.246101 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:46:12.246101 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:46:12.246101 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:46:12.246101 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:46:12.247285 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:46:12.247285 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:46:12.247285 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 14:46:12.247285 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 14:46:12.247285 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 14:46:12.247285 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 14:46:12.247285 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 14:46:12.247285 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 14:46:12.247285 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 01 14:46:12.247285 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Sep 01 14:46:12.247880 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:12.249742 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 287/593] 10.222.0.33 () {56 vars in 1102 bytes} [Thu Sep 1 14:46:12 2022] GET /identity/v3/users/neutron => generated 85 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:46:12.250997 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8461c857-d300-4eb6-8cad-3a0d9f17afa9 None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:12.274315 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14f2e7c-f39c-463f-a7bb-9901a8ad9371 None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:12.312660 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f683b901-c1d6-464f-afad-90fd16f1d71c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:12.313491 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f683b901-c1d6-464f-afad-90fd16f1d71c None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:12.313771 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f683b901-c1d6-464f-afad-90fd16f1d71c None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:12.314092 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f683b901-c1d6-464f-afad-90fd16f1d71c None admin] PATH_INFO: `/v3/users` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:12.315391 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f683b901-c1d6-464f-afad-90fd16f1d71c None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:46:12.315837 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f683b901-c1d6-464f-afad-90fd16f1d71c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:12.318113 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f683b901-c1d6-464f-afad-90fd16f1d71c None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:12.340588 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 307/594] 10.222.0.33 () {56 vars in 1137 bytes} [Thu Sep 1 14:46:12 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:46:12.341671 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f683b901-c1d6-464f-afad-90fd16f1d71c None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:12.391174 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8461c857-d300-4eb6-8cad-3a0d9f17afa9 None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:12.439193 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42f94ca-6e56-41e9-92fb-01fe24cce977 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:12.440709 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42f94ca-6e56-41e9-92fb-01fe24cce977 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:12.441199 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42f94ca-6e56-41e9-92fb-01fe24cce977 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:12.441772 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42f94ca-6e56-41e9-92fb-01fe24cce977 None admin] PATH_INFO: `/v3/projects` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:12.443313 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42f94ca-6e56-41e9-92fb-01fe24cce977 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81341) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:46:12.443618 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42f94ca-6e56-41e9-92fb-01fe24cce977 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:12.445054 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42f94ca-6e56-41e9-92fb-01fe24cce977 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:12.448969 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f683b901-c1d6-464f-afad-90fd16f1d71c None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:12.462663 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 288/595] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Sep 1 14:46:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:46:12.463548 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42f94ca-6e56-41e9-92fb-01fe24cce977 None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:12.486568 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61d1140-5a91-4830-9567-f9f20dbdbb31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:12.487428 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61d1140-5a91-4830-9567-f9f20dbdbb31 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:12.487709 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61d1140-5a91-4830-9567-f9f20dbdbb31 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:12.487984 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61d1140-5a91-4830-9567-f9f20dbdbb31 None admin] PATH_INFO: `/v3/users` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:12.488718 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61d1140-5a91-4830-9567-f9f20dbdbb31 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:46:12.489048 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61d1140-5a91-4830-9567-f9f20dbdbb31 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:12.490531 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61d1140-5a91-4830-9567-f9f20dbdbb31 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:12.493995 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:12.512723 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 308/596] 10.222.0.33 () {56 vars in 1139 bytes} [Thu Sep 1 14:46:12 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:46:12.513615 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61d1140-5a91-4830-9567-f9f20dbdbb31 None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:12.574640 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:12.595321 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42f94ca-6e56-41e9-92fb-01fe24cce977 None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:12.625840 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61d1140-5a91-4830-9567-f9f20dbdbb31 None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:12.631624 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fb4696-08d3-4c0d-a0c6-a44489da6b4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:12.632359 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fb4696-08d3-4c0d-a0c6-a44489da6b4a None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:12.632741 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fb4696-08d3-4c0d-a0c6-a44489da6b4a None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:12.633109 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fb4696-08d3-4c0d-a0c6-a44489da6b4a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:12.644621 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84fb4696-08d3-4c0d-a0c6-a44489da6b4a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:12.646648 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84fb4696-08d3-4c0d-a0c6-a44489da6b4a None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:12.662337 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application [None req-84fb4696-08d3-4c0d-a0c6-a44489da6b4a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 14:46:12.662337 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:46:12.662337 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:46:12.662337 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:46:12.662337 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:46:12.662337 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:46:12.662337 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:46:12.662337 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:46:12.662337 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:46:12.662337 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:46:12.662337 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:46:12.662337 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:46:12.662337 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 14:46:12.662337 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 14:46:12.662337 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 14:46:12.662337 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 14:46:12.663320 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:46:12.663320 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:46:12.663320 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:46:12.663320 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:46:12.663320 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:46:12.663320 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:46:12.663320 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:46:12.663320 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application with Lock( Sep 01 14:46:12.663320 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:46:12.663320 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:46:12.663320 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:46:12.663320 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:46:12.663320 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:46:12.663320 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:46:12.663320 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:46:12.663320 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:46:12.663320 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 14:46:12.664328 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 14:46:12.664328 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 14:46:12.664328 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Sep 01 14:46:12.664328 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 289/597] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 1 14:46:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:46:12.664328 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fb4696-08d3-4c0d-a0c6-a44489da6b4a None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:12.670887 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee9e131-bbba-4b47-a9af-e704e5502a81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:12.671548 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee9e131-bbba-4b47-a9af-e704e5502a81 None admin] REQUEST_METHOD: `PUT` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:12.671826 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee9e131-bbba-4b47-a9af-e704e5502a81 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:12.672094 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee9e131-bbba-4b47-a9af-e704e5502a81 None admin] PATH_INFO: `/v3/projects/e8c85b4bd17e4e89b56bf0f831b474d5/users/3069f72ce98049b99c9d98229a62bb51/roles/5e3f992e804046f381b3de8bf2a512e2` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:12.686307 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee9e131-bbba-4b47-a9af-e704e5502a81 None admin] RBAC: Authorizing `identity:create_grant(project_id=e8c85b4bd17e4e89b56bf0f831b474d5, user_id=3069f72ce98049b99c9d98229a62bb51, role_id=5e3f992e804046f381b3de8bf2a512e2)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:12.687757 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee9e131-bbba-4b47-a9af-e704e5502a81 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:12.714355 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 309/598] 10.222.0.33 () {58 vars in 1439 bytes} [Thu Sep 1 14:46:12 2022] PUT /identity/v3/projects/e8c85b4bd17e4e89b56bf0f831b474d5/users/3069f72ce98049b99c9d98229a62bb51/roles/5e3f992e804046f381b3de8bf2a512e2 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:46:12.714355 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace91fbb-82ce-45e8-a4e4-95f59ae04e5b None None] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:12.714355 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace91fbb-82ce-45e8-a4e4-95f59ae04e5b None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:12.714355 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace91fbb-82ce-45e8-a4e4-95f59ae04e5b None None] PATH_INFO: `/` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:12.714355 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 310/599] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 1 14:46:12 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:46:12.714355 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace91fbb-82ce-45e8-a4e4-95f59ae04e5b None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:12.785765 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:12.794387 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fb4696-08d3-4c0d-a0c6-a44489da6b4a None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:12.804086 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:12.838593 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22131025-88e2-4da8-85ef-54613784a5b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:12.838593 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22131025-88e2-4da8-85ef-54613784a5b7 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:12.838929 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22131025-88e2-4da8-85ef-54613784a5b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:12.839349 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22131025-88e2-4da8-85ef-54613784a5b7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:12.852924 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22131025-88e2-4da8-85ef-54613784a5b7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:12.854480 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22131025-88e2-4da8-85ef-54613784a5b7 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:12.861399 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace91fbb-82ce-45e8-a4e4-95f59ae04e5b None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:12.871222 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application [None req-22131025-88e2-4da8-85ef-54613784a5b7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 14:46:12.871222 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:46:12.871222 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:46:12.871222 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:46:12.871222 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:46:12.871222 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:46:12.871222 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:46:12.871222 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:46:12.871222 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:46:12.871222 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:46:12.871222 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:46:12.871222 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:46:12.871222 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 14:46:12.871222 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 14:46:12.871222 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 14:46:12.871222 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 14:46:12.872511 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:46:12.872511 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:46:12.872511 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:46:12.872511 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:46:12.872511 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:46:12.872511 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:46:12.872511 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:46:12.872511 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application with Lock( Sep 01 14:46:12.872511 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:46:12.872511 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:46:12.872511 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:46:12.872511 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:46:12.872511 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:46:12.872511 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:46:12.872511 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:46:12.872511 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:46:12.872511 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 14:46:12.873428 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 14:46:12.873428 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 14:46:12.873428 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Sep 01 14:46:12.873428 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 290/600] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 1 14:46:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:46:12.873663 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbebc69-f6ef-42e3-96fb-5b0be47a5082 None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:12.874191 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbebc69-f6ef-42e3-96fb-5b0be47a5082 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:12.874191 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbebc69-f6ef-42e3-96fb-5b0be47a5082 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:12.896757 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-7cbebc69-f6ef-42e3-96fb-5b0be47a5082 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:46:12.898595 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-7cbebc69-f6ef-42e3-96fb-5b0be47a5082 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:46:12.911874 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79309062-8f17-4322-a782-61d685628f64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:12.913531 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79309062-8f17-4322-a782-61d685628f64 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:12.914139 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79309062-8f17-4322-a782-61d685628f64 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:12.914691 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79309062-8f17-4322-a782-61d685628f64 None admin] PATH_INFO: `/v3/domains` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:12.915718 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79309062-8f17-4322-a782-61d685628f64 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:46:12.916195 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79309062-8f17-4322-a782-61d685628f64 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:12.918429 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79309062-8f17-4322-a782-61d685628f64 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:12.935056 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 311/601] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Sep 1 14:46:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:46:12.936181 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79309062-8f17-4322-a782-61d685628f64 None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:12.979163 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 291/602] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 1 14:46:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 14:46:12.980016 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbebc69-f6ef-42e3-96fb-5b0be47a5082 None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:13.058675 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79309062-8f17-4322-a782-61d685628f64 None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:13.062258 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:13.098051 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc976feb-fea7-4b6e-a7d8-535deb42ac40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:13.099734 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc976feb-fea7-4b6e-a7d8-535deb42ac40 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:13.100371 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc976feb-fea7-4b6e-a7d8-535deb42ac40 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:13.100836 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc976feb-fea7-4b6e-a7d8-535deb42ac40 None admin] PATH_INFO: `/v3/domains` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:13.102002 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc976feb-fea7-4b6e-a7d8-535deb42ac40 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:46:13.102660 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc976feb-fea7-4b6e-a7d8-535deb42ac40 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:13.104329 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc976feb-fea7-4b6e-a7d8-535deb42ac40 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:13.117683 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 312/603] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Sep 1 14:46:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:46:13.123856 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc976feb-fea7-4b6e-a7d8-535deb42ac40 None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:13.168986 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:13.202215 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbebc69-f6ef-42e3-96fb-5b0be47a5082 None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:13.235035 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc976feb-fea7-4b6e-a7d8-535deb42ac40 None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:13.271951 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f050fb-f00d-4512-9873-a377245bce24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:13.273070 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f050fb-f00d-4512-9873-a377245bce24 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:13.273550 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f050fb-f00d-4512-9873-a377245bce24 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:13.274032 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f050fb-f00d-4512-9873-a377245bce24 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:13.281530 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f050fb-f00d-4512-9873-a377245bce24 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:13.283357 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f050fb-f00d-4512-9873-a377245bce24 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:13.295526 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9e3a8b-d648-4bb2-af71-636ba7634dd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:13.296170 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9e3a8b-d648-4bb2-af71-636ba7634dd2 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:13.296356 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application [None req-e6f050fb-f00d-4512-9873-a377245bce24 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 14:46:13.296356 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:46:13.296356 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:46:13.296356 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:46:13.296356 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:46:13.296356 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:46:13.296356 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:46:13.296356 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:46:13.296356 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:46:13.296356 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:46:13.296356 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:46:13.296356 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:46:13.296356 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 14:46:13.296356 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 14:46:13.296356 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 14:46:13.297246 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 14:46:13.297246 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:46:13.297246 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:46:13.297246 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:46:13.297246 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:46:13.297246 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:46:13.297246 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:46:13.297246 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:46:13.297246 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application with Lock( Sep 01 14:46:13.297246 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:46:13.297246 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:46:13.297246 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:46:13.297246 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:46:13.297246 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:46:13.297246 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:46:13.297246 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:46:13.297246 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:46:13.298174 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 14:46:13.298174 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 14:46:13.298174 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 14:46:13.298174 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 14:46:13.298174 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 14:46:13.298174 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 14:46:13.298174 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 14:46:13.298174 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Sep 01 14:46:13.298644 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9e3a8b-d648-4bb2-af71-636ba7634dd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:13.298644 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9e3a8b-d648-4bb2-af71-636ba7634dd2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:13.300654 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 313/604] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Sep 1 14:46:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 14:46:13.306988 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f050fb-f00d-4512-9873-a377245bce24 None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:13.316561 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9e3a8b-d648-4bb2-af71-636ba7634dd2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:13.318132 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9e3a8b-d648-4bb2-af71-636ba7634dd2 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:13.334527 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:13.343438 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application [None req-ac9e3a8b-d648-4bb2-af71-636ba7634dd2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 14:46:13.343438 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:46:13.343438 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:46:13.343438 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:46:13.343438 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:46:13.343438 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:46:13.343438 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:46:13.343438 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:46:13.343438 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:46:13.343438 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:46:13.343438 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:46:13.343438 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:46:13.343438 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 14:46:13.343438 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 14:46:13.343438 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 14:46:13.344386 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 14:46:13.344386 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:46:13.344386 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:46:13.344386 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:46:13.344386 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:46:13.344386 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:46:13.344386 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:46:13.344386 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:46:13.344386 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application with Lock( Sep 01 14:46:13.344386 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:46:13.344386 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:46:13.344386 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:46:13.344386 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:46:13.344386 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:46:13.344386 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:46:13.344386 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:46:13.344386 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:46:13.345312 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 14:46:13.345312 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 14:46:13.345312 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 14:46:13.345312 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 14:46:13.345312 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 14:46:13.345312 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 14:46:13.345312 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 14:46:13.345312 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Sep 01 14:46:13.345312 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 292/605] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Sep 1 14:46:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 365 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:46:13.351129 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9e3a8b-d648-4bb2-af71-636ba7634dd2 None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:13.418630 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f050fb-f00d-4512-9873-a377245bce24 None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:13.453488 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b092b5ad-b430-49b3-9e70-6406ce51bd01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:13.454082 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b092b5ad-b430-49b3-9e70-6406ce51bd01 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:13.454393 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b092b5ad-b430-49b3-9e70-6406ce51bd01 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:13.454647 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b092b5ad-b430-49b3-9e70-6406ce51bd01 None admin] PATH_INFO: `/v3/projects` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:13.455355 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b092b5ad-b430-49b3-9e70-6406ce51bd01 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:46:13.455650 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b092b5ad-b430-49b3-9e70-6406ce51bd01 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:13.457057 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b092b5ad-b430-49b3-9e70-6406ce51bd01 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:13.468937 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 314/606] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Sep 1 14:46:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 14:46:13.475275 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622cc348-43d0-4e0f-add7-de132857763b None None] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:13.475555 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622cc348-43d0-4e0f-add7-de132857763b None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:13.475836 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622cc348-43d0-4e0f-add7-de132857763b None None] PATH_INFO: `/` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:13.477021 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 315/607] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 1 14:46:13 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:46:13.477694 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622cc348-43d0-4e0f-add7-de132857763b None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:13.514938 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9e3a8b-d648-4bb2-af71-636ba7634dd2 None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:13.550092 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e5977d-b1c5-419a-940f-b68b561f1068 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:13.550725 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e5977d-b1c5-419a-940f-b68b561f1068 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:13.551005 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e5977d-b1c5-419a-940f-b68b561f1068 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:13.551276 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e5977d-b1c5-419a-940f-b68b561f1068 None admin] PATH_INFO: `/v3/projects` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:13.552079 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e5977d-b1c5-419a-940f-b68b561f1068 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81341) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:46:13.552387 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e5977d-b1c5-419a-940f-b68b561f1068 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:13.553826 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e5977d-b1c5-419a-940f-b68b561f1068 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:13.566645 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 293/608] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Sep 1 14:46:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 14:46:13.567288 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e5977d-b1c5-419a-940f-b68b561f1068 None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:13.571423 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:13.578449 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:13.589797 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622cc348-43d0-4e0f-add7-de132857763b None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:13.625232 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0846d9-35a7-4143-824a-041bde1e8926 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:13.625859 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0846d9-35a7-4143-824a-041bde1e8926 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:13.626139 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0846d9-35a7-4143-824a-041bde1e8926 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:13.626440 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0846d9-35a7-4143-824a-041bde1e8926 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:13.637173 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0846d9-35a7-4143-824a-041bde1e8926 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 14:46:13.641977 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application [None req-8d0846d9-35a7-4143-824a-041bde1e8926 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 14:46:13.641977 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:46:13.641977 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:46:13.641977 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:46:13.641977 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:46:13.641977 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:46:13.641977 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:46:13.641977 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:46:13.641977 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:46:13.641977 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:46:13.641977 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:46:13.641977 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:46:13.641977 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 14:46:13.641977 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 14:46:13.641977 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 14:46:13.641977 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 14:46:13.642965 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 14:46:13.642965 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 14:46:13.642965 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 14:46:13.642965 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 14:46:13.642965 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:46:13.642965 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:46:13.642965 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:46:13.642965 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:46:13.642965 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:46:13.642965 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:46:13.642965 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:46:13.642965 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application with Lock( Sep 01 14:46:13.642965 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:46:13.642965 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:46:13.642965 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:46:13.642965 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:46:13.644442 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:46:13.644442 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:46:13.644442 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:46:13.644442 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:46:13.644442 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 14:46:13.644442 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 14:46:13.644442 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 14:46:13.644442 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 14:46:13.644442 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 14:46:13.644442 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 14:46:13.644442 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 14:46:13.644442 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Sep 01 14:46:13.644442 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 316/609] 10.222.0.33 () {56 vars in 1102 bytes} [Thu Sep 1 14:46:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:46:13.644442 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11034d2-35b7-4979-ba12-99f1d892bc0a None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:13.645895 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11034d2-35b7-4979-ba12-99f1d892bc0a None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:13.645895 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11034d2-35b7-4979-ba12-99f1d892bc0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:13.663857 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-e11034d2-35b7-4979-ba12-99f1d892bc0a None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:46:13.665062 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-e11034d2-35b7-4979-ba12-99f1d892bc0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:46:13.690053 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e5977d-b1c5-419a-940f-b68b561f1068 None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:13.727044 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c594ac5-727c-45d5-8ea8-bf33fc4cb64e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:13.727704 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c594ac5-727c-45d5-8ea8-bf33fc4cb64e None admin] REQUEST_METHOD: `PUT` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:13.727981 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c594ac5-727c-45d5-8ea8-bf33fc4cb64e None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:13.728250 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c594ac5-727c-45d5-8ea8-bf33fc4cb64e None admin] PATH_INFO: `/v3/projects/e8c85b4bd17e4e89b56bf0f831b474d5/users/2790149639aa418bbc4e4982e93eda44/roles/5e3f992e804046f381b3de8bf2a512e2` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:13.738433 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 317/610] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 1 14:46:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 14:46:13.738433 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11034d2-35b7-4979-ba12-99f1d892bc0a None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:13.749706 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c594ac5-727c-45d5-8ea8-bf33fc4cb64e None admin] RBAC: Authorizing `identity:create_grant(project_id=e8c85b4bd17e4e89b56bf0f831b474d5, user_id=2790149639aa418bbc4e4982e93eda44, role_id=5e3f992e804046f381b3de8bf2a512e2)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:13.751210 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c594ac5-727c-45d5-8ea8-bf33fc4cb64e None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:13.767438 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 294/611] 10.222.0.33 () {58 vars in 1439 bytes} [Thu Sep 1 14:46:13 2022] PUT /identity/v3/projects/e8c85b4bd17e4e89b56bf0f831b474d5/users/2790149639aa418bbc4e4982e93eda44/roles/5e3f992e804046f381b3de8bf2a512e2 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:46:13.768492 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c594ac5-727c-45d5-8ea8-bf33fc4cb64e None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:13.824354 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:13.856691 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11034d2-35b7-4979-ba12-99f1d892bc0a None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:13.857358 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:13.901835 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb9e200-55f1-40bc-90b8-2bf8dc36802e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:13.902964 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb9e200-55f1-40bc-90b8-2bf8dc36802e None admin] REQUEST_METHOD: `PUT` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:13.903457 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb9e200-55f1-40bc-90b8-2bf8dc36802e None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:13.903929 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb9e200-55f1-40bc-90b8-2bf8dc36802e None admin] PATH_INFO: `/v3/projects/e8c85b4bd17e4e89b56bf0f831b474d5/users/d3b81739e34e4e648f88b6d73fcb5e46/roles/5e3f992e804046f381b3de8bf2a512e2` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:13.919106 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c594ac5-727c-45d5-8ea8-bf33fc4cb64e None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:13.922966 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb9e200-55f1-40bc-90b8-2bf8dc36802e None admin] RBAC: Authorizing `identity:create_grant(project_id=e8c85b4bd17e4e89b56bf0f831b474d5, user_id=d3b81739e34e4e648f88b6d73fcb5e46, role_id=5e3f992e804046f381b3de8bf2a512e2)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:13.925264 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb9e200-55f1-40bc-90b8-2bf8dc36802e None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:13.952815 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 318/612] 10.222.0.33 () {58 vars in 1439 bytes} [Thu Sep 1 14:46:13 2022] PUT /identity/v3/projects/e8c85b4bd17e4e89b56bf0f831b474d5/users/d3b81739e34e4e648f88b6d73fcb5e46/roles/5e3f992e804046f381b3de8bf2a512e2 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:46:13.975618 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6362a93a-a7c1-496e-a28d-0890b13e29fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:13.976758 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6362a93a-a7c1-496e-a28d-0890b13e29fe None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:13.977311 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6362a93a-a7c1-496e-a28d-0890b13e29fe None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:13.977797 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6362a93a-a7c1-496e-a28d-0890b13e29fe None admin] PATH_INFO: `/v3/roles` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:13.978782 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6362a93a-a7c1-496e-a28d-0890b13e29fe None admin] RBAC: Adding query filter params (name=service) {{(pid=81341) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:46:13.979316 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6362a93a-a7c1-496e-a28d-0890b13e29fe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:13.981130 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6362a93a-a7c1-496e-a28d-0890b13e29fe None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:13.994151 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 295/613] 10.222.0.33 () {56 vars in 1103 bytes} [Thu Sep 1 14:46:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:46:14.001835 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb9e200-55f1-40bc-90b8-2bf8dc36802e None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:14.073034 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6362a93a-a7c1-496e-a28d-0890b13e29fe None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:14.154831 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb9e200-55f1-40bc-90b8-2bf8dc36802e None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:14.188405 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:14.205416 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa07a0a6-398f-4fd9-a1bf-aa2e3c91036f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:14.206049 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa07a0a6-398f-4fd9-a1bf-aa2e3c91036f None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:14.206355 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa07a0a6-398f-4fd9-a1bf-aa2e3c91036f None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:14.206634 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa07a0a6-398f-4fd9-a1bf-aa2e3c91036f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:14.212791 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6362a93a-a7c1-496e-a28d-0890b13e29fe None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:14.217746 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa07a0a6-398f-4fd9-a1bf-aa2e3c91036f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:14.219243 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa07a0a6-398f-4fd9-a1bf-aa2e3c91036f None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:14.234940 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application [None req-fa07a0a6-398f-4fd9-a1bf-aa2e3c91036f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 14:46:14.234940 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:46:14.234940 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:46:14.234940 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:46:14.234940 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:46:14.234940 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:46:14.234940 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:46:14.234940 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:46:14.234940 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:46:14.234940 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:46:14.234940 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:46:14.234940 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:46:14.234940 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 14:46:14.234940 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 14:46:14.234940 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 14:46:14.234940 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 14:46:14.235942 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:46:14.235942 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:46:14.235942 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:46:14.235942 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:46:14.235942 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:46:14.235942 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:46:14.235942 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:46:14.235942 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application with Lock( Sep 01 14:46:14.235942 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:46:14.235942 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:46:14.235942 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:46:14.235942 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:46:14.235942 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:46:14.235942 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:46:14.235942 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:46:14.235942 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:46:14.235942 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 14:46:14.236868 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 14:46:14.236868 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 14:46:14.236868 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Sep 01 14:46:14.236868 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 319/614] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 1 14:46:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:46:14.237789 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:14.242165 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa07a0a6-398f-4fd9-a1bf-aa2e3c91036f None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:14.255358 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406ffe31-250d-477e-a48e-273f358e2326 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:14.256218 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406ffe31-250d-477e-a48e-273f358e2326 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:14.256696 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406ffe31-250d-477e-a48e-273f358e2326 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:14.257108 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406ffe31-250d-477e-a48e-273f358e2326 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:14.268982 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406ffe31-250d-477e-a48e-273f358e2326 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 14:46:14.274651 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application [None req-406ffe31-250d-477e-a48e-273f358e2326 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 14:46:14.274651 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:46:14.274651 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:46:14.274651 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:46:14.274651 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:46:14.274651 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:46:14.274651 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:46:14.274651 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:46:14.274651 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:46:14.274651 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:46:14.274651 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:46:14.274651 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:46:14.274651 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 14:46:14.274651 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 14:46:14.274651 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 14:46:14.274651 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 14:46:14.275634 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 14:46:14.275634 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 14:46:14.275634 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 14:46:14.275634 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 14:46:14.275634 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:46:14.275634 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:46:14.275634 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:46:14.275634 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:46:14.275634 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:46:14.275634 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:46:14.275634 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:46:14.275634 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application with Lock( Sep 01 14:46:14.275634 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:46:14.275634 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:46:14.275634 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:46:14.275634 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:46:14.276487 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:46:14.276487 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:46:14.276487 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:46:14.276487 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:46:14.276487 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 14:46:14.276487 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 14:46:14.276487 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 14:46:14.276487 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 14:46:14.276487 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 14:46:14.276487 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 14:46:14.276487 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 14:46:14.276487 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Sep 01 14:46:14.278821 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 296/615] 10.222.0.33 () {56 vars in 1102 bytes} [Thu Sep 1 14:46:14 2022] GET /identity/v3/roles/service => generated 85 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:46:14.284811 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406ffe31-250d-477e-a48e-273f358e2326 None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:14.346387 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa07a0a6-398f-4fd9-a1bf-aa2e3c91036f None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:14.381457 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c9ef17-e8b4-41f8-8352-31cd492e5405 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:14.382065 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c9ef17-e8b4-41f8-8352-31cd492e5405 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:14.382370 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c9ef17-e8b4-41f8-8352-31cd492e5405 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:14.382636 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c9ef17-e8b4-41f8-8352-31cd492e5405 None admin] PATH_INFO: `/v3/domains` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:14.383275 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c9ef17-e8b4-41f8-8352-31cd492e5405 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:46:14.383645 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c9ef17-e8b4-41f8-8352-31cd492e5405 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:14.384985 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c9ef17-e8b4-41f8-8352-31cd492e5405 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:14.395061 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406ffe31-250d-477e-a48e-273f358e2326 None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:14.397386 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 320/616] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Sep 1 14:46:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:46:14.402763 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c9ef17-e8b4-41f8-8352-31cd492e5405 None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:14.432652 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68acf386-3324-463a-b2f5-597fe2de2ba5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:14.433802 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68acf386-3324-463a-b2f5-597fe2de2ba5 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:14.434330 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68acf386-3324-463a-b2f5-597fe2de2ba5 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:14.434868 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68acf386-3324-463a-b2f5-597fe2de2ba5 None admin] PATH_INFO: `/v3/roles` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:14.435842 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68acf386-3324-463a-b2f5-597fe2de2ba5 None admin] RBAC: Adding query filter params (name=service) {{(pid=81341) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:46:14.436347 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68acf386-3324-463a-b2f5-597fe2de2ba5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:14.438586 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:14.439555 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68acf386-3324-463a-b2f5-597fe2de2ba5 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:14.450598 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 297/617] 10.222.0.33 () {56 vars in 1103 bytes} [Thu Sep 1 14:46:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:46:14.456378 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68acf386-3324-463a-b2f5-597fe2de2ba5 None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:14.505805 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c9ef17-e8b4-41f8-8352-31cd492e5405 None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:14.540452 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37ecab0-afa4-4cfa-bdc7-15fcbfcaca3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:14.541063 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37ecab0-afa4-4cfa-bdc7-15fcbfcaca3a None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:14.541337 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37ecab0-afa4-4cfa-bdc7-15fcbfcaca3a None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:14.541602 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37ecab0-afa4-4cfa-bdc7-15fcbfcaca3a None admin] PATH_INFO: `/v3/users/swift` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:14.554374 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37ecab0-afa4-4cfa-bdc7-15fcbfcaca3a None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:14.554374 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37ecab0-afa4-4cfa-bdc7-15fcbfcaca3a None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:14.555198 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:14.573647 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application [None req-f37ecab0-afa4-4cfa-bdc7-15fcbfcaca3a None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 01 14:46:14.573647 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:46:14.573647 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:46:14.573647 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:46:14.573647 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:46:14.573647 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:46:14.573647 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:46:14.573647 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:46:14.573647 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:46:14.573647 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:46:14.573647 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:46:14.573647 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:46:14.573647 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 14:46:14.573647 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 14:46:14.573647 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 14:46:14.573647 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 14:46:14.574897 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:46:14.574897 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:46:14.574897 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 14:46:14.574897 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 14:46:14.574897 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 14:46:14.574897 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 14:46:14.574897 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:46:14.574897 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:46:14.574897 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:46:14.574897 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:46:14.574897 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:46:14.574897 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application with Lock( Sep 01 14:46:14.574897 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:46:14.574897 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:46:14.574897 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:46:14.574897 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:46:14.574897 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:46:14.574897 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:46:14.575951 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:46:14.575951 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:46:14.575951 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 14:46:14.575951 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 14:46:14.575951 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 14:46:14.575951 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 14:46:14.575951 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 14:46:14.575951 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 14:46:14.575951 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 01 14:46:14.575951 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Sep 01 14:46:14.584975 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 321/618] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Sep 1 14:46:14 2022] GET /identity/v3/users/swift => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 14:46:14.591508 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37ecab0-afa4-4cfa-bdc7-15fcbfcaca3a None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:14.641173 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68acf386-3324-463a-b2f5-597fe2de2ba5 None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:14.713535 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7423bff-ac15-45ac-86e3-bd228bb36add None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:14.714171 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7423bff-ac15-45ac-86e3-bd228bb36add None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:14.714470 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7423bff-ac15-45ac-86e3-bd228bb36add None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:14.714741 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7423bff-ac15-45ac-86e3-bd228bb36add None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:14.732893 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7423bff-ac15-45ac-86e3-bd228bb36add None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:14.734401 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7423bff-ac15-45ac-86e3-bd228bb36add None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:14.757088 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application [None req-e7423bff-ac15-45ac-86e3-bd228bb36add None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 14:46:14.757088 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:46:14.757088 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:46:14.757088 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:46:14.757088 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:46:14.757088 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:46:14.757088 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:46:14.757088 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:46:14.757088 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:46:14.757088 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:46:14.757088 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:46:14.757088 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:46:14.757088 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 14:46:14.757088 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 14:46:14.757088 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 14:46:14.757088 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 14:46:14.758263 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:46:14.758263 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:46:14.758263 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:46:14.758263 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:46:14.758263 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:46:14.758263 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:46:14.758263 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:46:14.758263 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application with Lock( Sep 01 14:46:14.758263 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:46:14.758263 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:46:14.758263 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:46:14.758263 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:46:14.758263 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:46:14.758263 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:46:14.758263 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:46:14.758263 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:46:14.758263 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 14:46:14.759186 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 14:46:14.759186 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 14:46:14.759186 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Sep 01 14:46:14.759186 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 298/619] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 1 14:46:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:46:14.763965 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7423bff-ac15-45ac-86e3-bd228bb36add None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:14.795711 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:14.806469 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37ecab0-afa4-4cfa-bdc7-15fcbfcaca3a None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:14.872205 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027aa845-d57b-4fbc-9946-6d60f83faa85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:14.872820 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027aa845-d57b-4fbc-9946-6d60f83faa85 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:14.873346 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027aa845-d57b-4fbc-9946-6d60f83faa85 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:14.873618 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027aa845-d57b-4fbc-9946-6d60f83faa85 None admin] PATH_INFO: `/v3/users` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:14.875050 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027aa845-d57b-4fbc-9946-6d60f83faa85 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:46:14.875364 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027aa845-d57b-4fbc-9946-6d60f83faa85 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:14.876799 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027aa845-d57b-4fbc-9946-6d60f83faa85 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:14.900154 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 322/620] 10.222.0.33 () {56 vars in 1135 bytes} [Thu Sep 1 14:46:14 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:46:14.902095 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fc6aed-5d6e-478d-a40d-45a912fa06c6 None None] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:14.902403 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fc6aed-5d6e-478d-a40d-45a912fa06c6 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:14.902680 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fc6aed-5d6e-478d-a40d-45a912fa06c6 None None] PATH_INFO: `/` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:14.904207 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 323/621] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 1 14:46:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:46:14.905813 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fc6aed-5d6e-478d-a40d-45a912fa06c6 None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:14.915516 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:14.925058 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7423bff-ac15-45ac-86e3-bd228bb36add None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:14.977496 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80fbf46-d62b-4eec-a5e7-d5858903548c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:14.978158 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80fbf46-d62b-4eec-a5e7-d5858903548c None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:14.978626 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80fbf46-d62b-4eec-a5e7-d5858903548c None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:14.979042 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80fbf46-d62b-4eec-a5e7-d5858903548c None admin] PATH_INFO: `/v3/domains` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:14.979750 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80fbf46-d62b-4eec-a5e7-d5858903548c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81341) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:46:14.980054 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80fbf46-d62b-4eec-a5e7-d5858903548c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:14.982171 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80fbf46-d62b-4eec-a5e7-d5858903548c None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:14.999212 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 299/622] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Sep 1 14:46:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 14:46:15.000448 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5684dd52-58a8-4184-bb66-4707ff792edd None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:15.000733 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5684dd52-58a8-4184-bb66-4707ff792edd None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:15.001009 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5684dd52-58a8-4184-bb66-4707ff792edd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:15.019962 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-5684dd52-58a8-4184-bb66-4707ff792edd None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:46:15.021910 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-5684dd52-58a8-4184-bb66-4707ff792edd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:46:15.050399 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fc6aed-5d6e-478d-a40d-45a912fa06c6 None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:15.099117 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 300/623] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 1 14:46:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 14:46:15.100367 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5684dd52-58a8-4184-bb66-4707ff792edd None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:15.112187 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3532d610-c6f4-486a-9446-8746b9e3ef60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:15.112916 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3532d610-c6f4-486a-9446-8746b9e3ef60 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:15.113301 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3532d610-c6f4-486a-9446-8746b9e3ef60 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:15.113673 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3532d610-c6f4-486a-9446-8746b9e3ef60 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:15.136195 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3532d610-c6f4-486a-9446-8746b9e3ef60 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:15.137832 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3532d610-c6f4-486a-9446-8746b9e3ef60 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:15.175881 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application [None req-3532d610-c6f4-486a-9446-8746b9e3ef60 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 14:46:15.175881 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:46:15.175881 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:46:15.175881 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:46:15.175881 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:46:15.175881 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:46:15.175881 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:46:15.175881 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:46:15.175881 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:46:15.175881 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:46:15.175881 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:46:15.175881 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:46:15.175881 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 14:46:15.175881 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 14:46:15.175881 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 14:46:15.175881 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 14:46:15.177103 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:46:15.177103 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:46:15.177103 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:46:15.177103 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:46:15.177103 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:46:15.177103 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:46:15.177103 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:46:15.177103 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application with Lock( Sep 01 14:46:15.177103 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:46:15.177103 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:46:15.177103 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:46:15.177103 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:46:15.177103 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:46:15.177103 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:46:15.177103 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:46:15.177103 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:46:15.177103 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 14:46:15.178208 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 14:46:15.178208 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 14:46:15.178208 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Sep 01 14:46:15.183489 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 324/624] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 1 14:46:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:46:15.188679 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3532d610-c6f4-486a-9446-8746b9e3ef60 None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:15.198372 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:15.208784 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:15.220137 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5684dd52-58a8-4184-bb66-4707ff792edd None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:15.259761 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591a2782-dafe-473f-ba7c-4243a100603a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:15.260599 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591a2782-dafe-473f-ba7c-4243a100603a None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:15.261018 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591a2782-dafe-473f-ba7c-4243a100603a None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:15.261501 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591a2782-dafe-473f-ba7c-4243a100603a None admin] PATH_INFO: `/v3/users/nova` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:15.269641 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591a2782-dafe-473f-ba7c-4243a100603a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:15.272419 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591a2782-dafe-473f-ba7c-4243a100603a None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:15.285568 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application [None req-591a2782-dafe-473f-ba7c-4243a100603a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 01 14:46:15.285568 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:46:15.285568 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:46:15.285568 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:46:15.285568 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:46:15.285568 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:46:15.285568 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:46:15.285568 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:46:15.285568 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:46:15.285568 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:46:15.285568 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:46:15.285568 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:46:15.285568 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 14:46:15.285568 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 14:46:15.285568 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 14:46:15.285568 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 14:46:15.287331 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:46:15.287331 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:46:15.287331 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 14:46:15.287331 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 14:46:15.287331 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 14:46:15.287331 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 14:46:15.287331 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:46:15.287331 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:46:15.287331 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:46:15.287331 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:46:15.287331 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:46:15.287331 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application with Lock( Sep 01 14:46:15.287331 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:46:15.287331 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:46:15.287331 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:46:15.287331 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:46:15.287331 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:46:15.287331 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:46:15.288523 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:46:15.288523 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:46:15.288523 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 14:46:15.288523 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 14:46:15.288523 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 14:46:15.288523 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 14:46:15.288523 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 14:46:15.288523 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 14:46:15.288523 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 01 14:46:15.288523 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Sep 01 14:46:15.292028 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 301/625] 10.222.0.33 () {56 vars in 1093 bytes} [Thu Sep 1 14:46:15 2022] GET /identity/v3/users/nova => generated 82 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:46:15.301329 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591a2782-dafe-473f-ba7c-4243a100603a None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:15.382932 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3532d610-c6f4-486a-9446-8746b9e3ef60 None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:15.428823 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591a2782-dafe-473f-ba7c-4243a100603a None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:15.446100 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d27c11-3cf3-4507-a7a9-a2a47333a6dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:15.446966 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d27c11-3cf3-4507-a7a9-a2a47333a6dc None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:15.447348 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d27c11-3cf3-4507-a7a9-a2a47333a6dc None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:15.447789 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d27c11-3cf3-4507-a7a9-a2a47333a6dc None admin] PATH_INFO: `/v3/domains` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:15.448547 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d27c11-3cf3-4507-a7a9-a2a47333a6dc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:46:15.448954 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d27c11-3cf3-4507-a7a9-a2a47333a6dc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:15.450559 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d27c11-3cf3-4507-a7a9-a2a47333a6dc None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:15.464408 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 325/626] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Sep 1 14:46:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:46:15.465213 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98efdb57-42e9-41e0-b598-af72a9adc85f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:15.465453 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d27c11-3cf3-4507-a7a9-a2a47333a6dc None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:15.465826 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98efdb57-42e9-41e0-b598-af72a9adc85f None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:15.466202 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98efdb57-42e9-41e0-b598-af72a9adc85f None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:15.466502 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98efdb57-42e9-41e0-b598-af72a9adc85f None admin] PATH_INFO: `/v3/users` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:15.467243 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98efdb57-42e9-41e0-b598-af72a9adc85f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81341) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:46:15.467541 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98efdb57-42e9-41e0-b598-af72a9adc85f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:15.469139 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98efdb57-42e9-41e0-b598-af72a9adc85f None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:15.490485 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 302/627] 10.222.0.33 () {56 vars in 1133 bytes} [Thu Sep 1 14:46:15 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:46:15.495861 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98efdb57-42e9-41e0-b598-af72a9adc85f None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:15.498874 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:15.538157 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:15.575204 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d27c11-3cf3-4507-a7a9-a2a47333a6dc None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:15.604486 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98efdb57-42e9-41e0-b598-af72a9adc85f None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:15.612147 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521ee337-dee8-4875-875c-35bf61d5dc68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:15.613183 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521ee337-dee8-4875-875c-35bf61d5dc68 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:15.613388 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521ee337-dee8-4875-875c-35bf61d5dc68 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:15.613762 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521ee337-dee8-4875-875c-35bf61d5dc68 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:15.625032 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521ee337-dee8-4875-875c-35bf61d5dc68 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 14:46:15.630917 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application [None req-521ee337-dee8-4875-875c-35bf61d5dc68 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 14:46:15.630917 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:46:15.630917 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:46:15.630917 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:46:15.630917 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:46:15.630917 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:46:15.630917 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:46:15.630917 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:46:15.630917 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:46:15.630917 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:46:15.630917 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:46:15.630917 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:46:15.630917 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 14:46:15.630917 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 14:46:15.630917 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 14:46:15.630917 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 14:46:15.632067 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 14:46:15.632067 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 14:46:15.632067 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 14:46:15.632067 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 14:46:15.632067 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:46:15.632067 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:46:15.632067 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:46:15.632067 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:46:15.632067 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:46:15.632067 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:46:15.632067 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:46:15.632067 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application with Lock( Sep 01 14:46:15.632067 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:46:15.632067 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:46:15.632067 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:46:15.632067 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:46:15.633113 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:46:15.633113 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:46:15.633113 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:46:15.633113 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:46:15.633113 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 14:46:15.633113 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 14:46:15.633113 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 14:46:15.633113 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 14:46:15.633113 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 14:46:15.633113 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 14:46:15.633113 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 14:46:15.633113 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Sep 01 14:46:15.639773 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 326/628] 10.222.0.33 () {56 vars in 1102 bytes} [Thu Sep 1 14:46:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:46:15.640592 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521ee337-dee8-4875-875c-35bf61d5dc68 None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:15.644972 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9764aeae-51b2-4318-9021-d259956059c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:15.646440 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9764aeae-51b2-4318-9021-d259956059c3 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:15.646863 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9764aeae-51b2-4318-9021-d259956059c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:15.647294 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9764aeae-51b2-4318-9021-d259956059c3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:15.654756 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9764aeae-51b2-4318-9021-d259956059c3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:15.656395 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9764aeae-51b2-4318-9021-d259956059c3 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:15.668694 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application [None req-9764aeae-51b2-4318-9021-d259956059c3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 14:46:15.668694 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:46:15.668694 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:46:15.668694 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:46:15.668694 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:46:15.668694 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:46:15.668694 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:46:15.668694 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:46:15.668694 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:46:15.668694 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:46:15.668694 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:46:15.668694 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:46:15.668694 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 14:46:15.668694 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 14:46:15.668694 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 14:46:15.669750 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 14:46:15.669750 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:46:15.669750 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:46:15.669750 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:46:15.669750 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:46:15.669750 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:46:15.669750 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:46:15.669750 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:46:15.669750 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application with Lock( Sep 01 14:46:15.669750 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:46:15.669750 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:46:15.669750 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:46:15.669750 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:46:15.669750 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:46:15.669750 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:46:15.669750 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:46:15.669750 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:46:15.670873 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 14:46:15.670873 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 14:46:15.670873 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 14:46:15.670873 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 14:46:15.670873 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 14:46:15.670873 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 14:46:15.670873 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 14:46:15.670873 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Sep 01 14:46:15.673522 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 303/629] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Sep 1 14:46:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:46:15.674632 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9764aeae-51b2-4318-9021-d259956059c3 None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:15.756066 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521ee337-dee8-4875-875c-35bf61d5dc68 None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:15.758544 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:15.784418 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:15.796041 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7068c44-6dee-47b5-b986-9bd0a1228a1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:15.797003 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7068c44-6dee-47b5-b986-9bd0a1228a1f None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:15.797517 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7068c44-6dee-47b5-b986-9bd0a1228a1f None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:15.797997 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7068c44-6dee-47b5-b986-9bd0a1228a1f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:15.817868 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9764aeae-51b2-4318-9021-d259956059c3 None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:15.858750 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d094b0-b9ac-4a63-b0cb-6db85ab7ab46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:15.859725 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d094b0-b9ac-4a63-b0cb-6db85ab7ab46 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:15.860215 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d094b0-b9ac-4a63-b0cb-6db85ab7ab46 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:15.860692 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d094b0-b9ac-4a63-b0cb-6db85ab7ab46 None admin] PATH_INFO: `/v3/roles` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:15.861562 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d094b0-b9ac-4a63-b0cb-6db85ab7ab46 None admin] RBAC: Adding query filter params (name=service) {{(pid=81341) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:46:15.862158 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d094b0-b9ac-4a63-b0cb-6db85ab7ab46 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:15.863810 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d094b0-b9ac-4a63-b0cb-6db85ab7ab46 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:15.873023 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 304/630] 10.222.0.33 () {56 vars in 1103 bytes} [Thu Sep 1 14:46:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:46:15.874163 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d094b0-b9ac-4a63-b0cb-6db85ab7ab46 None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:15.886516 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7068c44-6dee-47b5-b986-9bd0a1228a1f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:15.888494 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7068c44-6dee-47b5-b986-9bd0a1228a1f None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:15.994571 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d094b0-b9ac-4a63-b0cb-6db85ab7ab46 None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:16.008955 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 327/631] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Sep 1 14:46:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 14:46:16.010160 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7068c44-6dee-47b5-b986-9bd0a1228a1f None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:16.023671 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:16.037184 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51f3e8c-04d5-4959-bd25-aeb557cb19a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:16.037859 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51f3e8c-04d5-4959-bd25-aeb557cb19a6 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:16.038229 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51f3e8c-04d5-4959-bd25-aeb557cb19a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:16.038582 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51f3e8c-04d5-4959-bd25-aeb557cb19a6 None admin] PATH_INFO: `/v3/projects` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:16.039166 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:16.039369 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51f3e8c-04d5-4959-bd25-aeb557cb19a6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81341) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:46:16.039686 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51f3e8c-04d5-4959-bd25-aeb557cb19a6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:16.041277 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51f3e8c-04d5-4959-bd25-aeb557cb19a6 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:16.059094 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 305/632] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Sep 1 14:46:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:46:16.059094 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c2ba7f-6f10-48e7-9d2c-3a9ccf4345a2 None None] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:16.059094 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c2ba7f-6f10-48e7-9d2c-3a9ccf4345a2 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:16.059094 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c2ba7f-6f10-48e7-9d2c-3a9ccf4345a2 None None] PATH_INFO: `/` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:16.059094 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 306/633] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 1 14:46:16 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:46:16.059094 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c2ba7f-6f10-48e7-9d2c-3a9ccf4345a2 None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:16.127523 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7068c44-6dee-47b5-b986-9bd0a1228a1f None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:16.162363 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c2ba7f-6f10-48e7-9d2c-3a9ccf4345a2 None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:16.163229 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f79b1f-bb9b-4385-929f-453ba6b90da3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:16.163930 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f79b1f-bb9b-4385-929f-453ba6b90da3 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:16.164305 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f79b1f-bb9b-4385-929f-453ba6b90da3 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:16.164644 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f79b1f-bb9b-4385-929f-453ba6b90da3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:16.175757 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f79b1f-bb9b-4385-929f-453ba6b90da3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:16.178053 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f79b1f-bb9b-4385-929f-453ba6b90da3 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:16.192971 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application [None req-f9f79b1f-bb9b-4385-929f-453ba6b90da3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 14:46:16.192971 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:46:16.192971 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:46:16.192971 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:46:16.192971 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:46:16.192971 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:46:16.192971 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:46:16.192971 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:46:16.192971 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:46:16.192971 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:46:16.192971 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:46:16.192971 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:46:16.192971 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 14:46:16.192971 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 14:46:16.192971 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 14:46:16.192971 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 14:46:16.194106 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:46:16.194106 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:46:16.194106 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:46:16.194106 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:46:16.194106 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:46:16.194106 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:46:16.194106 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:46:16.194106 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application with Lock( Sep 01 14:46:16.194106 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:46:16.194106 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:46:16.194106 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:46:16.194106 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:46:16.194106 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:46:16.194106 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:46:16.194106 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:46:16.194106 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:46:16.194106 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 14:46:16.195247 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 14:46:16.195247 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 14:46:16.195247 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Sep 01 14:46:16.200851 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 328/634] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 1 14:46:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:46:16.201616 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f79b1f-bb9b-4385-929f-453ba6b90da3 None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:16.201770 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d3287e-f288-40c3-90e0-5c9b6d114907 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:16.202321 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d3287e-f288-40c3-90e0-5c9b6d114907 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:16.202601 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d3287e-f288-40c3-90e0-5c9b6d114907 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:16.202867 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d3287e-f288-40c3-90e0-5c9b6d114907 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:16.214391 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d3287e-f288-40c3-90e0-5c9b6d114907 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:16.216198 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d3287e-f288-40c3-90e0-5c9b6d114907 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:16.232261 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application [None req-c4d3287e-f288-40c3-90e0-5c9b6d114907 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 14:46:16.232261 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:46:16.232261 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:46:16.232261 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:46:16.232261 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:46:16.232261 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:46:16.232261 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:46:16.232261 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:46:16.232261 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:46:16.232261 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:46:16.232261 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:46:16.232261 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:46:16.232261 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 14:46:16.232261 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 14:46:16.232261 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 14:46:16.232261 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 14:46:16.233285 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:46:16.233285 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:46:16.233285 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:46:16.233285 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:46:16.233285 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:46:16.233285 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:46:16.233285 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:46:16.233285 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application with Lock( Sep 01 14:46:16.233285 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:46:16.233285 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:46:16.233285 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:46:16.233285 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:46:16.233285 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:46:16.233285 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:46:16.233285 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:46:16.233285 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:46:16.233285 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 14:46:16.234210 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 14:46:16.234210 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 14:46:16.234210 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Sep 01 14:46:16.234210 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 307/635] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 1 14:46:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:46:16.234757 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d548f3-fd67-4505-a4d6-f0129023526b None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:16.234850 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d548f3-fd67-4505-a4d6-f0129023526b None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:16.235066 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d548f3-fd67-4505-a4d6-f0129023526b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:16.255063 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-16d548f3-fd67-4505-a4d6-f0129023526b None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:46:16.256240 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-16d548f3-fd67-4505-a4d6-f0129023526b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:46:16.285110 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:16.293341 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:16.316244 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f79b1f-bb9b-4385-929f-453ba6b90da3 None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:16.330042 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 308/636] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 1 14:46:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 14:46:16.331023 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d548f3-fd67-4505-a4d6-f0129023526b None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:16.351685 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3df3c55-3d16-42c0-8256-b0b4df3461b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:16.352464 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3df3c55-3d16-42c0-8256-b0b4df3461b7 None admin] REQUEST_METHOD: `PUT` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:16.352873 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3df3c55-3d16-42c0-8256-b0b4df3461b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:16.353246 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3df3c55-3d16-42c0-8256-b0b4df3461b7 None admin] PATH_INFO: `/v3/projects/e8c85b4bd17e4e89b56bf0f831b474d5/users/73a5bda5a6ea46e9863ccafd57dee339/roles/5e3f992e804046f381b3de8bf2a512e2` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:16.368372 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3df3c55-3d16-42c0-8256-b0b4df3461b7 None admin] RBAC: Authorizing `identity:create_grant(project_id=e8c85b4bd17e4e89b56bf0f831b474d5, user_id=73a5bda5a6ea46e9863ccafd57dee339, role_id=5e3f992e804046f381b3de8bf2a512e2)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:16.369858 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3df3c55-3d16-42c0-8256-b0b4df3461b7 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:16.385670 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 329/637] 10.222.0.33 () {58 vars in 1439 bytes} [Thu Sep 1 14:46:16 2022] PUT /identity/v3/projects/e8c85b4bd17e4e89b56bf0f831b474d5/users/73a5bda5a6ea46e9863ccafd57dee339/roles/5e3f992e804046f381b3de8bf2a512e2 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:46:16.386866 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3df3c55-3d16-42c0-8256-b0b4df3461b7 None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:16.458750 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d548f3-fd67-4505-a4d6-f0129023526b None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:16.500134 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0518821-0a8c-42a4-9101-d3724bdc6c83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:16.501114 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0518821-0a8c-42a4-9101-d3724bdc6c83 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:16.501530 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0518821-0a8c-42a4-9101-d3724bdc6c83 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:16.502008 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0518821-0a8c-42a4-9101-d3724bdc6c83 None admin] PATH_INFO: `/v3/domains` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:16.502872 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0518821-0a8c-42a4-9101-d3724bdc6c83 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81341) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:46:16.503318 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0518821-0a8c-42a4-9101-d3724bdc6c83 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:16.505249 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0518821-0a8c-42a4-9101-d3724bdc6c83 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:16.519279 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 309/638] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Sep 1 14:46:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:46:16.521534 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e584d1f5-da6b-4d70-b820-37ad5692332b None None] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:16.521966 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e584d1f5-da6b-4d70-b820-37ad5692332b None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:16.522442 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e584d1f5-da6b-4d70-b820-37ad5692332b None None] PATH_INFO: `/` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:16.524743 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 310/639] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 1 14:46:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:46:16.527257 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e584d1f5-da6b-4d70-b820-37ad5692332b None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:16.529328 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3df3c55-3d16-42c0-8256-b0b4df3461b7 None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:16.563514 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:16.566116 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:16.581676 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2dc1dd3-4101-4eba-a099-3f7df1c4c2e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:16.582590 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dc1dd3-4101-4eba-a099-3f7df1c4c2e1 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:16.583835 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dc1dd3-4101-4eba-a099-3f7df1c4c2e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:16.584252 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dc1dd3-4101-4eba-a099-3f7df1c4c2e1 None admin] PATH_INFO: `/v3/domains` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:16.585192 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2dc1dd3-4101-4eba-a099-3f7df1c4c2e1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:46:16.585638 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2dc1dd3-4101-4eba-a099-3f7df1c4c2e1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:16.587798 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2dc1dd3-4101-4eba-a099-3f7df1c4c2e1 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:16.603616 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 330/640] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Sep 1 14:46:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:46:16.605768 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf2a278-1fbc-4b8f-ac06-dc68158fcb35 None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:16.606223 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf2a278-1fbc-4b8f-ac06-dc68158fcb35 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:16.606716 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf2a278-1fbc-4b8f-ac06-dc68158fcb35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:16.629184 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-edf2a278-1fbc-4b8f-ac06-dc68158fcb35 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:46:16.630464 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-edf2a278-1fbc-4b8f-ac06-dc68158fcb35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:46:16.644742 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e584d1f5-da6b-4d70-b820-37ad5692332b None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:16.689070 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71fda8f-c4b2-4519-bc70-1a2965c67e74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:16.689947 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71fda8f-c4b2-4519-bc70-1a2965c67e74 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:16.690227 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71fda8f-c4b2-4519-bc70-1a2965c67e74 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:16.690547 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71fda8f-c4b2-4519-bc70-1a2965c67e74 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:16.698629 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71fda8f-c4b2-4519-bc70-1a2965c67e74 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:16.700166 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71fda8f-c4b2-4519-bc70-1a2965c67e74 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:16.710302 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 331/641] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 1 14:46:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 14:46:16.710302 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf2a278-1fbc-4b8f-ac06-dc68158fcb35 None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:16.716109 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application [None req-f71fda8f-c4b2-4519-bc70-1a2965c67e74 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 01 14:46:16.716109 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:46:16.716109 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:46:16.716109 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:46:16.716109 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:46:16.716109 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:46:16.716109 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:46:16.716109 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:46:16.716109 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:46:16.716109 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:46:16.716109 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:46:16.716109 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:46:16.716109 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 14:46:16.716109 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 14:46:16.716109 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 14:46:16.716109 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 14:46:16.717274 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:46:16.717274 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:46:16.717274 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 14:46:16.717274 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 14:46:16.717274 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 14:46:16.717274 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 14:46:16.717274 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:46:16.717274 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:46:16.717274 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:46:16.717274 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:46:16.717274 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:46:16.717274 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application with Lock( Sep 01 14:46:16.717274 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:46:16.717274 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:46:16.717274 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:46:16.717274 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:46:16.717274 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:46:16.717274 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:46:16.718498 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:46:16.718498 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:46:16.718498 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 14:46:16.718498 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 14:46:16.718498 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 14:46:16.718498 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 14:46:16.718498 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 14:46:16.718498 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 14:46:16.718498 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 01 14:46:16.718498 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Sep 01 14:46:16.726137 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 311/642] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Sep 1 14:46:16 2022] GET /identity/v3/users/glance => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:46:16.727209 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71fda8f-c4b2-4519-bc70-1a2965c67e74 None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:16.825933 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:16.839001 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71fda8f-c4b2-4519-bc70-1a2965c67e74 None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:16.846478 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf2a278-1fbc-4b8f-ac06-dc68158fcb35 None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:16.876375 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:16.882009 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353797f6-26e5-4f3c-88d7-ee1ac138e3ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:16.882679 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353797f6-26e5-4f3c-88d7-ee1ac138e3ad None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:16.882958 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353797f6-26e5-4f3c-88d7-ee1ac138e3ad None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:16.883233 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353797f6-26e5-4f3c-88d7-ee1ac138e3ad None admin] PATH_INFO: `/v3/roles/service` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:16.895032 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353797f6-26e5-4f3c-88d7-ee1ac138e3ad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 14:46:16.895269 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aac6d64-d1a1-4de0-8f63-0861a6618230 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:16.896336 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aac6d64-d1a1-4de0-8f63-0861a6618230 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:16.896606 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aac6d64-d1a1-4de0-8f63-0861a6618230 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:16.896867 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aac6d64-d1a1-4de0-8f63-0861a6618230 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:16.901101 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application [None req-353797f6-26e5-4f3c-88d7-ee1ac138e3ad None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 14:46:16.901101 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:46:16.901101 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:46:16.901101 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:46:16.901101 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:46:16.901101 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:46:16.901101 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:46:16.901101 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:46:16.901101 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:46:16.901101 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:46:16.901101 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:46:16.901101 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:46:16.901101 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 14:46:16.901101 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 14:46:16.901101 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 14:46:16.901101 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 14:46:16.902119 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 14:46:16.902119 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 14:46:16.902119 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 14:46:16.902119 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 14:46:16.902119 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:46:16.902119 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:46:16.902119 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:46:16.902119 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:46:16.902119 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:46:16.902119 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:46:16.902119 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:46:16.902119 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application with Lock( Sep 01 14:46:16.902119 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:46:16.902119 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:46:16.902119 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:46:16.902119 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:46:16.903969 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:46:16.903969 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:46:16.903969 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:46:16.903969 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:46:16.903969 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 14:46:16.903969 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 14:46:16.903969 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 14:46:16.903969 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 14:46:16.903969 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 14:46:16.903969 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 14:46:16.903969 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 14:46:16.903969 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Sep 01 14:46:16.903969 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 312/643] 10.222.0.33 () {56 vars in 1102 bytes} [Thu Sep 1 14:46:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:46:16.903969 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353797f6-26e5-4f3c-88d7-ee1ac138e3ad None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:16.904755 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aac6d64-d1a1-4de0-8f63-0861a6618230 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:16.905721 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aac6d64-d1a1-4de0-8f63-0861a6618230 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:16.920062 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application [None req-7aac6d64-d1a1-4de0-8f63-0861a6618230 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 14:46:16.920062 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:46:16.920062 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:46:16.920062 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:46:16.920062 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:46:16.920062 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:46:16.920062 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:46:16.920062 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:46:16.920062 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:46:16.920062 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:46:16.920062 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:46:16.920062 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:46:16.920062 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 14:46:16.920062 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 14:46:16.920062 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 14:46:16.921690 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 14:46:16.921690 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:46:16.921690 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:46:16.921690 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:46:16.921690 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:46:16.921690 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:46:16.921690 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:46:16.921690 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:46:16.921690 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application with Lock( Sep 01 14:46:16.921690 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:46:16.921690 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:46:16.921690 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:46:16.921690 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:46:16.921690 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:46:16.921690 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:46:16.921690 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:46:16.921690 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:46:16.923788 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 14:46:16.923788 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 14:46:16.923788 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 14:46:16.923788 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 14:46:16.923788 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 14:46:16.923788 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 14:46:16.923788 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 14:46:16.923788 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Sep 01 14:46:16.923788 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 332/644] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Sep 1 14:46:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:46:16.923788 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aac6d64-d1a1-4de0-8f63-0861a6618230 None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:17.015206 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353797f6-26e5-4f3c-88d7-ee1ac138e3ad None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:17.026063 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aac6d64-d1a1-4de0-8f63-0861a6618230 None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:17.051073 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a3891a-e74e-4906-99c9-c553d199242a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:17.051709 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a3891a-e74e-4906-99c9-c553d199242a None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:17.051990 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a3891a-e74e-4906-99c9-c553d199242a None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:17.052258 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a3891a-e74e-4906-99c9-c553d199242a None admin] PATH_INFO: `/v3/users` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:17.052987 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a3891a-e74e-4906-99c9-c553d199242a None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81341) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:46:17.053287 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a3891a-e74e-4906-99c9-c553d199242a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:17.054776 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a3891a-e74e-4906-99c9-c553d199242a None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:17.066376 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3451100-31d7-4237-bb26-820be52fc379 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:17.067852 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3451100-31d7-4237-bb26-820be52fc379 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:17.068317 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3451100-31d7-4237-bb26-820be52fc379 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:17.068772 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3451100-31d7-4237-bb26-820be52fc379 None admin] PATH_INFO: `/v3/roles` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:17.070140 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3451100-31d7-4237-bb26-820be52fc379 None admin] RBAC: Adding query filter params (name=service) {{(pid=81342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:46:17.070941 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3451100-31d7-4237-bb26-820be52fc379 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:17.072734 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3451100-31d7-4237-bb26-820be52fc379 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:17.073202 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 313/645] 10.222.0.33 () {56 vars in 1137 bytes} [Thu Sep 1 14:46:16 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:46:17.074032 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a3891a-e74e-4906-99c9-c553d199242a None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:17.081491 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:17.090611 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 333/646] 10.222.0.33 () {56 vars in 1103 bytes} [Thu Sep 1 14:46:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:46:17.090611 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3451100-31d7-4237-bb26-820be52fc379 None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:17.139687 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:17.187345 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a3891a-e74e-4906-99c9-c553d199242a None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:17.202731 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3451100-31d7-4237-bb26-820be52fc379 None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:17.224620 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b77926-9cd4-457d-8120-2a5290bfb9f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:17.225215 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b77926-9cd4-457d-8120-2a5290bfb9f1 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:17.225489 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b77926-9cd4-457d-8120-2a5290bfb9f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:17.225767 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b77926-9cd4-457d-8120-2a5290bfb9f1 None admin] PATH_INFO: `/v3/projects` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:17.226457 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b77926-9cd4-457d-8120-2a5290bfb9f1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81341) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:46:17.226970 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b77926-9cd4-457d-8120-2a5290bfb9f1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:17.228180 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b77926-9cd4-457d-8120-2a5290bfb9f1 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:17.237619 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f0324a-f697-44f2-9f55-ff3a29f15f65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:17.238260 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f0324a-f697-44f2-9f55-ff3a29f15f65 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:17.238562 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f0324a-f697-44f2-9f55-ff3a29f15f65 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:17.238874 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f0324a-f697-44f2-9f55-ff3a29f15f65 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:17.240398 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 314/647] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Sep 1 14:46:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:46:17.241011 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b77926-9cd4-457d-8120-2a5290bfb9f1 None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:17.255308 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f0324a-f697-44f2-9f55-ff3a29f15f65 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 14:46:17.260821 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application [None req-b3f0324a-f697-44f2-9f55-ff3a29f15f65 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 14:46:17.260821 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:46:17.260821 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:46:17.260821 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:46:17.260821 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:46:17.260821 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:46:17.260821 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:46:17.260821 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:46:17.260821 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:46:17.260821 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:46:17.260821 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:46:17.260821 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:46:17.260821 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 14:46:17.260821 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 14:46:17.260821 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 14:46:17.260821 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 14:46:17.261993 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 14:46:17.261993 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 14:46:17.261993 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 14:46:17.261993 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 14:46:17.261993 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:46:17.261993 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:46:17.261993 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:46:17.261993 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:46:17.261993 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:46:17.261993 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:46:17.261993 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:46:17.261993 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application with Lock( Sep 01 14:46:17.261993 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:46:17.261993 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:46:17.261993 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:46:17.261993 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:46:17.262962 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:46:17.262962 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:46:17.262962 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:46:17.262962 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:46:17.262962 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 14:46:17.262962 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 14:46:17.262962 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 14:46:17.262962 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 14:46:17.262962 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 14:46:17.262962 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 14:46:17.262962 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 14:46:17.262962 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Sep 01 14:46:17.263649 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 334/648] 10.222.0.33 () {56 vars in 1102 bytes} [Thu Sep 1 14:46:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:46:17.264190 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f0324a-f697-44f2-9f55-ff3a29f15f65 None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:17.303030 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:17.345019 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b77926-9cd4-457d-8120-2a5290bfb9f1 None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:17.367393 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:17.370143 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f0324a-f697-44f2-9f55-ff3a29f15f65 None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:17.381657 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bff19b-5184-44d3-9bcb-c024029379f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:17.382371 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bff19b-5184-44d3-9bcb-c024029379f0 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:17.382568 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bff19b-5184-44d3-9bcb-c024029379f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:17.382844 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bff19b-5184-44d3-9bcb-c024029379f0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:17.407553 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f1cac7-f9a5-49c6-b19a-62ad2f90ab71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:17.408173 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f1cac7-f9a5-49c6-b19a-62ad2f90ab71 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:17.408446 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f1cac7-f9a5-49c6-b19a-62ad2f90ab71 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:17.408738 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f1cac7-f9a5-49c6-b19a-62ad2f90ab71 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:17.420090 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f1cac7-f9a5-49c6-b19a-62ad2f90ab71 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:17.421571 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f1cac7-f9a5-49c6-b19a-62ad2f90ab71 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:17.436857 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application [None req-92f1cac7-f9a5-49c6-b19a-62ad2f90ab71 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 14:46:17.436857 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:46:17.436857 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:46:17.436857 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:46:17.436857 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:46:17.436857 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:46:17.436857 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:46:17.436857 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:46:17.436857 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:46:17.436857 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:46:17.436857 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:46:17.436857 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:46:17.436857 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 14:46:17.436857 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 14:46:17.436857 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 14:46:17.436857 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 14:46:17.437855 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:46:17.437855 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:46:17.437855 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:46:17.437855 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:46:17.437855 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:46:17.437855 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:46:17.437855 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:46:17.437855 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application with Lock( Sep 01 14:46:17.437855 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:46:17.437855 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:46:17.437855 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:46:17.437855 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:46:17.437855 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:46:17.437855 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:46:17.437855 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:46:17.437855 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:46:17.437855 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 14:46:17.438834 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 14:46:17.438834 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 14:46:17.438834 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Sep 01 14:46:17.443327 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 335/649] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 1 14:46:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:46:17.443960 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f1cac7-f9a5-49c6-b19a-62ad2f90ab71 None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:17.455798 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bff19b-5184-44d3-9bcb-c024029379f0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:17.457637 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bff19b-5184-44d3-9bcb-c024029379f0 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:17.526466 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:17.554402 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f1cac7-f9a5-49c6-b19a-62ad2f90ab71 None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:17.567927 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 315/650] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Sep 1 14:46:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 14:46:17.569036 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bff19b-5184-44d3-9bcb-c024029379f0 None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:17.595014 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ef90bf-1e53-429a-9432-60463dc9cedb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:17.595675 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ef90bf-1e53-429a-9432-60463dc9cedb None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:17.595897 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ef90bf-1e53-429a-9432-60463dc9cedb None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:17.596167 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ef90bf-1e53-429a-9432-60463dc9cedb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:17.596895 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ef90bf-1e53-429a-9432-60463dc9cedb None admin] RBAC: Adding query filter params (role.id=5e3f992e804046f381b3de8bf2a512e2, scope.project.id=e8c85b4bd17e4e89b56bf0f831b474d5, user.id=4858bb8621854ec8aba836e7d9050498) {{(pid=81342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:46:17.597207 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ef90bf-1e53-429a-9432-60463dc9cedb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:17.599203 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ef90bf-1e53-429a-9432-60463dc9cedb None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:17.604840 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 336/651] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Sep 1 14:46:17 2022] GET /identity/v3/role_assignments?user.id=4858bb8621854ec8aba836e7d9050498&scope.project.id=e8c85b4bd17e4e89b56bf0f831b474d5&role.id=5e3f992e804046f381b3de8bf2a512e2 => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:46:17.605617 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ef90bf-1e53-429a-9432-60463dc9cedb None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:17.633599 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:17.711775 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bff19b-5184-44d3-9bcb-c024029379f0 None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:17.729060 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ef90bf-1e53-429a-9432-60463dc9cedb None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:17.760950 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64703c56-8a22-4aaa-a33d-09e7aef7ff26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:17.762649 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64703c56-8a22-4aaa-a33d-09e7aef7ff26 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:17.763123 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64703c56-8a22-4aaa-a33d-09e7aef7ff26 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:17.763792 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64703c56-8a22-4aaa-a33d-09e7aef7ff26 None admin] PATH_INFO: `/v3/roles` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:17.764732 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64703c56-8a22-4aaa-a33d-09e7aef7ff26 None admin] RBAC: Adding query filter params (name=service) {{(pid=81341) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:46:17.765278 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64703c56-8a22-4aaa-a33d-09e7aef7ff26 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:17.766952 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64703c56-8a22-4aaa-a33d-09e7aef7ff26 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:17.767809 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad82f72-3274-4896-b182-ad0465e6e597 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:17.768531 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad82f72-3274-4896-b182-ad0465e6e597 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:17.768897 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad82f72-3274-4896-b182-ad0465e6e597 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:17.769260 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad82f72-3274-4896-b182-ad0465e6e597 None admin] PATH_INFO: `/v3/domains` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:17.769979 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad82f72-3274-4896-b182-ad0465e6e597 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:46:17.770418 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad82f72-3274-4896-b182-ad0465e6e597 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:17.771829 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad82f72-3274-4896-b182-ad0465e6e597 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:17.779368 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 316/652] 10.222.0.33 () {56 vars in 1103 bytes} [Thu Sep 1 14:46:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:46:17.780553 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64703c56-8a22-4aaa-a33d-09e7aef7ff26 None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:17.788199 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 337/653] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Sep 1 14:46:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:46:17.789033 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad82f72-3274-4896-b182-ad0465e6e597 None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:17.819564 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:17.899173 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:17.912713 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64703c56-8a22-4aaa-a33d-09e7aef7ff26 None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:17.931077 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad82f72-3274-4896-b182-ad0465e6e597 None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:17.948796 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8dcde2-ceff-40d4-ab58-b305de2a4788 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:17.949806 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8dcde2-ceff-40d4-ab58-b305de2a4788 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:17.950324 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8dcde2-ceff-40d4-ab58-b305de2a4788 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:17.950816 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8dcde2-ceff-40d4-ab58-b305de2a4788 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:17.965064 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8dcde2-ceff-40d4-ab58-b305de2a4788 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:17.966840 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8dcde2-ceff-40d4-ab58-b305de2a4788 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:17.976989 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c110b8c2-54e1-4b43-9efa-64640a87e1ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:17.977885 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c110b8c2-54e1-4b43-9efa-64640a87e1ee None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:17.978405 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c110b8c2-54e1-4b43-9efa-64640a87e1ee None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:17.978880 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c110b8c2-54e1-4b43-9efa-64640a87e1ee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:17.985001 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application [None req-ce8dcde2-ceff-40d4-ab58-b305de2a4788 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 14:46:17.985001 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:46:17.985001 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:46:17.985001 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:46:17.985001 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:46:17.985001 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:46:17.985001 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:46:17.985001 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:46:17.985001 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:46:17.985001 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:46:17.985001 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:46:17.985001 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:46:17.985001 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 14:46:17.985001 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 14:46:17.985001 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 14:46:17.985001 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 14:46:17.985975 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:46:17.985975 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:46:17.985975 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:46:17.985975 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:46:17.985975 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:46:17.985975 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:46:17.985975 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:46:17.985975 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application with Lock( Sep 01 14:46:17.985975 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:46:17.985975 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:46:17.985975 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:46:17.985975 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:46:17.985975 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:46:17.985975 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:46:17.985975 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:46:17.985975 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:46:17.985975 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 14:46:17.987298 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 14:46:17.987298 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 14:46:17.987298 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Sep 01 14:46:17.990322 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 317/654] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 1 14:46:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:46:17.993240 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8dcde2-ceff-40d4-ab58-b305de2a4788 None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:17.995318 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c110b8c2-54e1-4b43-9efa-64640a87e1ee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:17.997024 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c110b8c2-54e1-4b43-9efa-64640a87e1ee None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:18.013472 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application [None req-c110b8c2-54e1-4b43-9efa-64640a87e1ee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 14:46:18.013472 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:46:18.013472 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:46:18.013472 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:46:18.013472 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:46:18.013472 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:46:18.013472 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:46:18.013472 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:46:18.013472 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:46:18.013472 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:46:18.013472 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:46:18.013472 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:46:18.013472 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 14:46:18.013472 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 14:46:18.013472 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 14:46:18.013472 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 14:46:18.014547 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:46:18.014547 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:46:18.014547 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:46:18.014547 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:46:18.014547 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:46:18.014547 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:46:18.014547 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:46:18.014547 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application with Lock( Sep 01 14:46:18.014547 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:46:18.014547 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:46:18.014547 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:46:18.014547 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:46:18.014547 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:46:18.014547 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:46:18.014547 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:46:18.014547 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:46:18.014547 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 14:46:18.015466 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 14:46:18.015466 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 14:46:18.015466 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Sep 01 14:46:18.017489 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 338/655] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 1 14:46:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:46:18.018637 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c110b8c2-54e1-4b43-9efa-64640a87e1ee None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:18.106645 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:18.129388 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c110b8c2-54e1-4b43-9efa-64640a87e1ee None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:18.129697 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8dcde2-ceff-40d4-ab58-b305de2a4788 None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:18.159208 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:18.169492 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759fdb0b-a1fd-4bfa-8519-d6b03ce83781 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:18.170452 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759fdb0b-a1fd-4bfa-8519-d6b03ce83781 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:18.170939 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759fdb0b-a1fd-4bfa-8519-d6b03ce83781 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:18.171412 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759fdb0b-a1fd-4bfa-8519-d6b03ce83781 None admin] PATH_INFO: `/v3/domains` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:18.172260 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759fdb0b-a1fd-4bfa-8519-d6b03ce83781 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:46:18.172789 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759fdb0b-a1fd-4bfa-8519-d6b03ce83781 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:18.174355 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a96676-7834-4b3d-82fc-a68d91dee5d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:18.174726 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759fdb0b-a1fd-4bfa-8519-d6b03ce83781 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:18.174986 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a96676-7834-4b3d-82fc-a68d91dee5d6 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:18.175262 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a96676-7834-4b3d-82fc-a68d91dee5d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:18.175530 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a96676-7834-4b3d-82fc-a68d91dee5d6 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:18.184835 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a96676-7834-4b3d-82fc-a68d91dee5d6 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:18.187193 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a96676-7834-4b3d-82fc-a68d91dee5d6 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:18.188804 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 339/656] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Sep 1 14:46:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:46:18.189841 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759fdb0b-a1fd-4bfa-8519-d6b03ce83781 None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:18.206945 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application [None req-e2a96676-7834-4b3d-82fc-a68d91dee5d6 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 01 14:46:18.206945 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:46:18.206945 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:46:18.206945 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:46:18.206945 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:46:18.206945 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:46:18.206945 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:46:18.206945 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:46:18.206945 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:46:18.206945 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:46:18.206945 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:46:18.206945 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:46:18.206945 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 14:46:18.206945 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 14:46:18.206945 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 14:46:18.206945 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 14:46:18.208511 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:46:18.208511 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:46:18.208511 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 14:46:18.208511 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 14:46:18.208511 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 14:46:18.208511 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 14:46:18.208511 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:46:18.208511 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:46:18.208511 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:46:18.208511 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:46:18.208511 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:46:18.208511 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application with Lock( Sep 01 14:46:18.208511 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:46:18.208511 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:46:18.208511 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:46:18.208511 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:46:18.208511 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:46:18.208511 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:46:18.209493 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:46:18.209493 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:46:18.209493 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 14:46:18.209493 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 14:46:18.209493 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 14:46:18.209493 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 14:46:18.209493 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 14:46:18.209493 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 14:46:18.209493 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 01 14:46:18.209493 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Sep 01 14:46:18.209493 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 318/657] 10.222.0.33 () {56 vars in 1102 bytes} [Thu Sep 1 14:46:17 2022] GET /identity/v3/users/neutron => generated 85 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:46:18.214643 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a96676-7834-4b3d-82fc-a68d91dee5d6 None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:18.303122 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759fdb0b-a1fd-4bfa-8519-d6b03ce83781 None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:18.341095 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878b0bb2-7400-4447-aeb2-3bc5ee71b14f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:18.341593 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a96676-7834-4b3d-82fc-a68d91dee5d6 None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:18.341933 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878b0bb2-7400-4447-aeb2-3bc5ee71b14f None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:18.342332 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878b0bb2-7400-4447-aeb2-3bc5ee71b14f None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:18.342748 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878b0bb2-7400-4447-aeb2-3bc5ee71b14f None admin] PATH_INFO: `/v3/domains` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:18.343559 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878b0bb2-7400-4447-aeb2-3bc5ee71b14f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:46:18.344008 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878b0bb2-7400-4447-aeb2-3bc5ee71b14f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:18.345465 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878b0bb2-7400-4447-aeb2-3bc5ee71b14f None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:18.358063 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 340/658] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Sep 1 14:46:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:46:18.359030 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878b0bb2-7400-4447-aeb2-3bc5ee71b14f None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:18.382856 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199d9ba9-f247-443a-a5f5-a41ffcefc190 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:18.384002 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199d9ba9-f247-443a-a5f5-a41ffcefc190 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:18.384277 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199d9ba9-f247-443a-a5f5-a41ffcefc190 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:18.384812 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199d9ba9-f247-443a-a5f5-a41ffcefc190 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:18.393985 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199d9ba9-f247-443a-a5f5-a41ffcefc190 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:18.397124 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199d9ba9-f247-443a-a5f5-a41ffcefc190 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:18.398950 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:18.410439 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application [None req-199d9ba9-f247-443a-a5f5-a41ffcefc190 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 14:46:18.410439 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:46:18.410439 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:46:18.410439 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:46:18.410439 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:46:18.410439 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:46:18.410439 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:46:18.410439 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:46:18.410439 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:46:18.410439 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:46:18.410439 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:46:18.410439 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:46:18.410439 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 14:46:18.410439 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 14:46:18.410439 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 14:46:18.411412 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 14:46:18.411412 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:46:18.411412 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:46:18.411412 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:46:18.411412 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:46:18.411412 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:46:18.411412 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:46:18.411412 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:46:18.411412 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application with Lock( Sep 01 14:46:18.411412 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:46:18.411412 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:46:18.411412 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:46:18.411412 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:46:18.411412 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:46:18.411412 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:46:18.411412 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:46:18.411412 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:46:18.412900 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 14:46:18.412900 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 14:46:18.412900 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 14:46:18.412900 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 14:46:18.412900 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 14:46:18.412900 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 14:46:18.412900 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 14:46:18.412900 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Sep 01 14:46:18.412900 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 319/659] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Sep 1 14:46:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:46:18.412900 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199d9ba9-f247-443a-a5f5-a41ffcefc190 None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:18.423944 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:18.483958 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878b0bb2-7400-4447-aeb2-3bc5ee71b14f None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:18.516071 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199d9ba9-f247-443a-a5f5-a41ffcefc190 None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:18.518459 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8293e8ee-7351-42dc-a433-1351acbee724 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:18.519068 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8293e8ee-7351-42dc-a433-1351acbee724 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:18.519351 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8293e8ee-7351-42dc-a433-1351acbee724 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:18.519624 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8293e8ee-7351-42dc-a433-1351acbee724 None admin] PATH_INFO: `/v3/users` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:18.520912 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8293e8ee-7351-42dc-a433-1351acbee724 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:46:18.521226 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8293e8ee-7351-42dc-a433-1351acbee724 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:18.522689 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8293e8ee-7351-42dc-a433-1351acbee724 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:18.546150 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 341/660] 10.222.0.33 () {56 vars in 1139 bytes} [Thu Sep 1 14:46:18 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:46:18.546150 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8293e8ee-7351-42dc-a433-1351acbee724 None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:18.554476 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b67016-13a0-4173-abfb-e96e109ada53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:18.555111 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b67016-13a0-4173-abfb-e96e109ada53 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:18.555419 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b67016-13a0-4173-abfb-e96e109ada53 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:18.555692 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b67016-13a0-4173-abfb-e96e109ada53 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:18.563543 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b67016-13a0-4173-abfb-e96e109ada53 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:18.564995 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b67016-13a0-4173-abfb-e96e109ada53 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:18.577087 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application [None req-28b67016-13a0-4173-abfb-e96e109ada53 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 01 14:46:18.577087 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:46:18.577087 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:46:18.577087 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:46:18.577087 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:46:18.577087 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:46:18.577087 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:46:18.577087 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:46:18.577087 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:46:18.577087 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:46:18.577087 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:46:18.577087 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:46:18.577087 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 14:46:18.577087 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 14:46:18.577087 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 14:46:18.577087 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 14:46:18.578120 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:46:18.578120 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:46:18.578120 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 14:46:18.578120 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 14:46:18.578120 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 14:46:18.578120 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 14:46:18.578120 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:46:18.578120 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:46:18.578120 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:46:18.578120 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:46:18.578120 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:46:18.578120 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application with Lock( Sep 01 14:46:18.578120 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:46:18.578120 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:46:18.578120 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:46:18.578120 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:46:18.578120 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:46:18.578120 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:46:18.579154 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:46:18.579154 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:46:18.579154 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 14:46:18.579154 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 14:46:18.579154 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 14:46:18.579154 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 14:46:18.579154 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 14:46:18.579154 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 14:46:18.579154 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 01 14:46:18.579154 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Sep 01 14:46:18.579154 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 320/661] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Sep 1 14:46:18 2022] GET /identity/v3/users/cinder => generated 84 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:46:18.579774 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcc98b3-d63e-4025-b96e-c91ee5f5503e None None] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:18.579774 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcc98b3-d63e-4025-b96e-c91ee5f5503e None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:18.579774 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcc98b3-d63e-4025-b96e-c91ee5f5503e None None] PATH_INFO: `/` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:18.582285 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 321/662] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 1 14:46:18 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:46:18.582285 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcc98b3-d63e-4025-b96e-c91ee5f5503e None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:18.644336 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:18.664463 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8293e8ee-7351-42dc-a433-1351acbee724 None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:18.672204 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:18.695973 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcc98b3-d63e-4025-b96e-c91ee5f5503e None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:18.701827 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0235fb-811c-4323-a7d5-f8168debafc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:18.702469 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0235fb-811c-4323-a7d5-f8168debafc4 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:18.702751 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0235fb-811c-4323-a7d5-f8168debafc4 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:18.703021 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0235fb-811c-4323-a7d5-f8168debafc4 None admin] PATH_INFO: `/v3/projects` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:18.703713 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0235fb-811c-4323-a7d5-f8168debafc4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:46:18.704020 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0235fb-811c-4323-a7d5-f8168debafc4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:18.705442 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0235fb-811c-4323-a7d5-f8168debafc4 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:18.718498 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 342/663] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Sep 1 14:46:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:46:18.724032 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0235fb-811c-4323-a7d5-f8168debafc4 None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:18.733674 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dac70c-15a7-4b36-b501-c6ce02ea7703 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:18.734585 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dac70c-15a7-4b36-b501-c6ce02ea7703 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:18.735508 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dac70c-15a7-4b36-b501-c6ce02ea7703 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:18.735995 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dac70c-15a7-4b36-b501-c6ce02ea7703 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:18.805668 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23dac70c-15a7-4b36-b501-c6ce02ea7703 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:18.807170 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23dac70c-15a7-4b36-b501-c6ce02ea7703 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:18.838221 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0235fb-811c-4323-a7d5-f8168debafc4 None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:18.866916 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:18.877019 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61583a9e-15ac-4179-918f-dac4407edae4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:18.877801 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61583a9e-15ac-4179-918f-dac4407edae4 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:18.878072 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61583a9e-15ac-4179-918f-dac4407edae4 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:18.878363 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61583a9e-15ac-4179-918f-dac4407edae4 None admin] PATH_INFO: `/v3/users` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:18.879084 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61583a9e-15ac-4179-918f-dac4407edae4 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:46:18.879378 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61583a9e-15ac-4179-918f-dac4407edae4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:18.881343 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61583a9e-15ac-4179-918f-dac4407edae4 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:18.897880 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 343/664] 10.222.0.33 () {56 vars in 1137 bytes} [Thu Sep 1 14:46:18 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:46:18.899151 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08347c66-9c89-4b22-b322-9ef84ebfc87b None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:18.899443 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08347c66-9c89-4b22-b322-9ef84ebfc87b None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:18.899723 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08347c66-9c89-4b22-b322-9ef84ebfc87b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:18.916985 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-08347c66-9c89-4b22-b322-9ef84ebfc87b None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:46:18.918084 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-08347c66-9c89-4b22-b322-9ef84ebfc87b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:46:18.923291 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 322/665] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Sep 1 14:46:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 14:46:18.923291 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dac70c-15a7-4b36-b501-c6ce02ea7703 None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:18.942793 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:19.010289 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 344/666] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 1 14:46:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 14:46:19.010289 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08347c66-9c89-4b22-b322-9ef84ebfc87b None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:19.029205 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dac70c-15a7-4b36-b501-c6ce02ea7703 None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:19.065936 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b91eb38-c106-4473-b3a5-5a77cae16fc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:19.066896 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b91eb38-c106-4473-b3a5-5a77cae16fc9 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:19.067082 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b91eb38-c106-4473-b3a5-5a77cae16fc9 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:19.067788 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b91eb38-c106-4473-b3a5-5a77cae16fc9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:19.068595 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b91eb38-c106-4473-b3a5-5a77cae16fc9 None admin] RBAC: Adding query filter params (role.id=5e3f992e804046f381b3de8bf2a512e2, scope.project.id=e8c85b4bd17e4e89b56bf0f831b474d5, user.id=3069f72ce98049b99c9d98229a62bb51) {{(pid=81341) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:46:19.068800 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b91eb38-c106-4473-b3a5-5a77cae16fc9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:19.070249 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b91eb38-c106-4473-b3a5-5a77cae16fc9 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:19.082267 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 323/667] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Sep 1 14:46:18 2022] GET /identity/v3/role_assignments?user.id=3069f72ce98049b99c9d98229a62bb51&scope.project.id=e8c85b4bd17e4e89b56bf0f831b474d5&role.id=5e3f992e804046f381b3de8bf2a512e2 => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:46:19.082267 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b91eb38-c106-4473-b3a5-5a77cae16fc9 None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:19.133243 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:19.183107 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08347c66-9c89-4b22-b322-9ef84ebfc87b None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:19.249069 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29da9802-a418-425d-a471-be332faf20b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:19.249715 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29da9802-a418-425d-a471-be332faf20b4 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:19.249992 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29da9802-a418-425d-a471-be332faf20b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:19.250279 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29da9802-a418-425d-a471-be332faf20b4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:19.278772 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:19.286720 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b91eb38-c106-4473-b3a5-5a77cae16fc9 None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:19.329635 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e88dbb9-9758-48fc-9beb-ca21f95ed008 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:19.330618 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e88dbb9-9758-48fc-9beb-ca21f95ed008 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:19.331209 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e88dbb9-9758-48fc-9beb-ca21f95ed008 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:19.331452 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e88dbb9-9758-48fc-9beb-ca21f95ed008 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:19.338010 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29da9802-a418-425d-a471-be332faf20b4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:19.339575 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29da9802-a418-425d-a471-be332faf20b4 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:19.347933 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e88dbb9-9758-48fc-9beb-ca21f95ed008 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:19.349843 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e88dbb9-9758-48fc-9beb-ca21f95ed008 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:19.368358 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application [None req-4e88dbb9-9758-48fc-9beb-ca21f95ed008 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 14:46:19.368358 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:46:19.368358 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:46:19.368358 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:46:19.368358 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:46:19.368358 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:46:19.368358 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:46:19.368358 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:46:19.368358 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:46:19.368358 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:46:19.368358 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:46:19.368358 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:46:19.368358 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 14:46:19.368358 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 14:46:19.368358 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 14:46:19.368358 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 14:46:19.369431 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:46:19.369431 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:46:19.369431 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:46:19.369431 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:46:19.369431 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:46:19.369431 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:46:19.369431 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:46:19.369431 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application with Lock( Sep 01 14:46:19.369431 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:46:19.369431 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:46:19.369431 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:46:19.369431 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:46:19.369431 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:46:19.369431 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:46:19.369431 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:46:19.369431 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:46:19.369431 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 14:46:19.370794 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 14:46:19.370794 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 14:46:19.370794 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Sep 01 14:46:19.370794 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 324/668] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 1 14:46:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:46:19.371026 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e88dbb9-9758-48fc-9beb-ca21f95ed008 None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:19.462352 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:19.481032 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 345/669] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Sep 1 14:46:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 478 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 14:46:19.482274 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29da9802-a418-425d-a471-be332faf20b4 None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:19.490301 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e88dbb9-9758-48fc-9beb-ca21f95ed008 None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:19.529053 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c5bca9-3fd2-4d0b-8261-54d88c4369d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:19.530049 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c5bca9-3fd2-4d0b-8261-54d88c4369d2 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:19.530550 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c5bca9-3fd2-4d0b-8261-54d88c4369d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:19.531018 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c5bca9-3fd2-4d0b-8261-54d88c4369d2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:19.543030 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c5bca9-3fd2-4d0b-8261-54d88c4369d2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 14:46:19.548662 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application [None req-56c5bca9-3fd2-4d0b-8261-54d88c4369d2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 14:46:19.548662 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:46:19.548662 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:46:19.548662 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:46:19.548662 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:46:19.548662 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:46:19.548662 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:46:19.548662 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:46:19.548662 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:46:19.548662 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:46:19.548662 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:46:19.548662 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:46:19.548662 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 14:46:19.548662 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 14:46:19.548662 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 14:46:19.548662 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 14:46:19.549634 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 14:46:19.549634 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 14:46:19.549634 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 14:46:19.549634 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 14:46:19.549634 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:46:19.549634 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:46:19.549634 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:46:19.549634 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:46:19.549634 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:46:19.549634 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:46:19.549634 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:46:19.549634 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application with Lock( Sep 01 14:46:19.549634 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:46:19.549634 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:46:19.549634 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:46:19.549634 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:46:19.550544 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:46:19.550544 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:46:19.550544 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:46:19.550544 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:46:19.550544 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 14:46:19.550544 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 14:46:19.550544 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 14:46:19.550544 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 14:46:19.550544 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 14:46:19.550544 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 14:46:19.550544 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 14:46:19.550544 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Sep 01 14:46:19.551218 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:19.553780 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 325/670] 10.222.0.33 () {56 vars in 1102 bytes} [Thu Sep 1 14:46:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:46:19.554822 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c5bca9-3fd2-4d0b-8261-54d88c4369d2 None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:19.604151 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29da9802-a418-425d-a471-be332faf20b4 None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:19.649279 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba30064-0000-4feb-946d-3da26865026b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:19.649875 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba30064-0000-4feb-946d-3da26865026b None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:19.650151 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba30064-0000-4feb-946d-3da26865026b None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:19.650445 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba30064-0000-4feb-946d-3da26865026b None admin] PATH_INFO: `/v3/domains` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:19.651202 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba30064-0000-4feb-946d-3da26865026b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:46:19.651503 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba30064-0000-4feb-946d-3da26865026b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:19.653339 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba30064-0000-4feb-946d-3da26865026b None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:19.670394 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 346/671] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Sep 1 14:46:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:46:19.671512 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba30064-0000-4feb-946d-3da26865026b None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:19.685226 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c5bca9-3fd2-4d0b-8261-54d88c4369d2 None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:19.720774 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6bdd5c-4001-4a76-b8fa-d48a4ea38f95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:19.721468 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6bdd5c-4001-4a76-b8fa-d48a4ea38f95 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:19.721748 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6bdd5c-4001-4a76-b8fa-d48a4ea38f95 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:19.722014 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6bdd5c-4001-4a76-b8fa-d48a4ea38f95 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:19.725105 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:19.736310 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6bdd5c-4001-4a76-b8fa-d48a4ea38f95 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:19.737769 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6bdd5c-4001-4a76-b8fa-d48a4ea38f95 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:19.751997 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application [None req-ec6bdd5c-4001-4a76-b8fa-d48a4ea38f95 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 14:46:19.751997 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:46:19.751997 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:46:19.751997 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:46:19.751997 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:46:19.751997 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:46:19.751997 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:46:19.751997 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:46:19.751997 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:46:19.751997 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:46:19.751997 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:46:19.751997 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:46:19.751997 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 14:46:19.751997 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 14:46:19.751997 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 14:46:19.751997 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 14:46:19.753131 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:46:19.753131 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:46:19.753131 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:46:19.753131 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:46:19.753131 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:46:19.753131 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:46:19.753131 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:46:19.753131 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application with Lock( Sep 01 14:46:19.753131 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:46:19.753131 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:46:19.753131 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:46:19.753131 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:46:19.753131 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:46:19.753131 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:46:19.753131 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:46:19.753131 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:46:19.753131 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 14:46:19.754341 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 14:46:19.754341 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 14:46:19.754341 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Sep 01 14:46:19.757618 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 326/672] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 1 14:46:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:46:19.758471 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6bdd5c-4001-4a76-b8fa-d48a4ea38f95 None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:19.791296 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba30064-0000-4feb-946d-3da26865026b None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:19.805747 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:19.829330 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7368d6-bfbc-49a8-a768-7c9773e1924e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:19.829939 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7368d6-bfbc-49a8-a768-7c9773e1924e None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:19.830213 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7368d6-bfbc-49a8-a768-7c9773e1924e None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:19.830669 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7368d6-bfbc-49a8-a768-7c9773e1924e None admin] PATH_INFO: `/v3/roles` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:19.831326 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7368d6-bfbc-49a8-a768-7c9773e1924e None admin] RBAC: Adding query filter params (name=service) {{(pid=81342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:46:19.832106 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7368d6-bfbc-49a8-a768-7c9773e1924e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:19.834024 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7368d6-bfbc-49a8-a768-7c9773e1924e None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:19.846494 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 347/673] 10.222.0.33 () {56 vars in 1103 bytes} [Thu Sep 1 14:46:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:46:19.846494 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02e9ca6-d66c-4546-8a36-c2041edaabf3 None None] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:19.846494 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02e9ca6-d66c-4546-8a36-c2041edaabf3 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:19.846494 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02e9ca6-d66c-4546-8a36-c2041edaabf3 None None] PATH_INFO: `/` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:19.846494 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 348/674] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 1 14:46:19 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:46:19.846494 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02e9ca6-d66c-4546-8a36-c2041edaabf3 None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:19.874385 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6bdd5c-4001-4a76-b8fa-d48a4ea38f95 None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:19.911460 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063ec0aa-848b-4750-a091-dd84baa5dc1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:19.912076 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063ec0aa-848b-4750-a091-dd84baa5dc1b None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:19.912352 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063ec0aa-848b-4750-a091-dd84baa5dc1b None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:19.912704 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063ec0aa-848b-4750-a091-dd84baa5dc1b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:19.919720 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063ec0aa-848b-4750-a091-dd84baa5dc1b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:19.921203 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063ec0aa-848b-4750-a091-dd84baa5dc1b None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:19.932463 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application [None req-063ec0aa-848b-4750-a091-dd84baa5dc1b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 14:46:19.932463 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:46:19.932463 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:46:19.932463 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:46:19.932463 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:46:19.932463 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:46:19.932463 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:46:19.932463 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:46:19.932463 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:46:19.932463 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:46:19.932463 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:46:19.932463 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:46:19.932463 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 14:46:19.932463 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 14:46:19.932463 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 14:46:19.933559 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 14:46:19.933559 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:46:19.933559 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:46:19.933559 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:46:19.933559 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:46:19.933559 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:46:19.933559 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:46:19.933559 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:46:19.933559 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application with Lock( Sep 01 14:46:19.933559 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:46:19.933559 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:46:19.933559 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:46:19.933559 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:46:19.933559 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:46:19.933559 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:46:19.933559 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:46:19.933559 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:46:19.934941 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 14:46:19.934941 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 14:46:19.934941 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 14:46:19.934941 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 14:46:19.934941 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 14:46:19.934941 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 14:46:19.934941 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 14:46:19.934941 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Sep 01 14:46:19.938232 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 327/675] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Sep 1 14:46:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:46:19.939572 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063ec0aa-848b-4750-a091-dd84baa5dc1b None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:19.965534 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02e9ca6-d66c-4546-8a36-c2041edaabf3 None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:20.001905 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611062e5-b845-47f4-99e4-78e35ff90695 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:20.002808 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611062e5-b845-47f4-99e4-78e35ff90695 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:20.003320 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611062e5-b845-47f4-99e4-78e35ff90695 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:20.003803 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611062e5-b845-47f4-99e4-78e35ff90695 None admin] PATH_INFO: `/v3/domains` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:20.005187 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611062e5-b845-47f4-99e4-78e35ff90695 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:46:20.005670 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611062e5-b845-47f4-99e4-78e35ff90695 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:20.007284 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611062e5-b845-47f4-99e4-78e35ff90695 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:20.021684 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:20.021858 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 349/676] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Sep 1 14:46:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:46:20.023040 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb585760-ee8f-4c32-8a3c-21c608602b01 None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:20.023469 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb585760-ee8f-4c32-8a3c-21c608602b01 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:20.023817 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb585760-ee8f-4c32-8a3c-21c608602b01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:20.041745 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-fb585760-ee8f-4c32-8a3c-21c608602b01 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:46:20.042961 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-fb585760-ee8f-4c32-8a3c-21c608602b01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:46:20.075955 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:20.101111 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063ec0aa-848b-4750-a091-dd84baa5dc1b None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:20.123038 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 350/677] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 1 14:46:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 14:46:20.124381 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb585760-ee8f-4c32-8a3c-21c608602b01 None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:20.136594 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435fc6f8-ad84-4c0c-a75c-28da79773f15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:20.137214 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435fc6f8-ad84-4c0c-a75c-28da79773f15 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:20.137488 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435fc6f8-ad84-4c0c-a75c-28da79773f15 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:20.137755 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435fc6f8-ad84-4c0c-a75c-28da79773f15 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:20.149122 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435fc6f8-ad84-4c0c-a75c-28da79773f15 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:20.150607 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435fc6f8-ad84-4c0c-a75c-28da79773f15 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:20.166914 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application [None req-435fc6f8-ad84-4c0c-a75c-28da79773f15 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 14:46:20.166914 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:46:20.166914 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:46:20.166914 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:46:20.166914 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:46:20.166914 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:46:20.166914 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:46:20.166914 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:46:20.166914 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:46:20.166914 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:46:20.166914 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:46:20.166914 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:46:20.166914 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 14:46:20.166914 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 14:46:20.166914 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 14:46:20.166914 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 14:46:20.167968 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:46:20.167968 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:46:20.167968 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:46:20.167968 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:46:20.167968 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:46:20.167968 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:46:20.167968 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:46:20.167968 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application with Lock( Sep 01 14:46:20.167968 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:46:20.167968 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:46:20.167968 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:46:20.167968 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:46:20.167968 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:46:20.167968 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:46:20.167968 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:46:20.167968 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:46:20.167968 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 14:46:20.168987 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 14:46:20.168987 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 14:46:20.168987 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Sep 01 14:46:20.168987 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 328/678] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 1 14:46:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:46:20.169229 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435fc6f8-ad84-4c0c-a75c-28da79773f15 None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:20.260728 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb585760-ee8f-4c32-8a3c-21c608602b01 None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:20.277199 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:20.281286 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435fc6f8-ad84-4c0c-a75c-28da79773f15 None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:20.310685 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14234a8-b160-4828-9e53-5c985f74441d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:20.311796 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14234a8-b160-4828-9e53-5c985f74441d None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:20.312240 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14234a8-b160-4828-9e53-5c985f74441d None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:20.312914 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14234a8-b160-4828-9e53-5c985f74441d None admin] PATH_INFO: `/v3/projects` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:20.314173 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14234a8-b160-4828-9e53-5c985f74441d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:46:20.314891 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14234a8-b160-4828-9e53-5c985f74441d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:20.316490 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14234a8-b160-4828-9e53-5c985f74441d None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:20.319082 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbe687a-e7c4-4b31-be4d-4158f9f0730d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:20.319797 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbe687a-e7c4-4b31-be4d-4158f9f0730d None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:20.320686 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbe687a-e7c4-4b31-be4d-4158f9f0730d None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:20.320686 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbe687a-e7c4-4b31-be4d-4158f9f0730d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:20.327856 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbe687a-e7c4-4b31-be4d-4158f9f0730d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:20.329411 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbe687a-e7c4-4b31-be4d-4158f9f0730d None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:20.332712 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 351/679] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Sep 1 14:46:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:46:20.333805 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14234a8-b160-4828-9e53-5c985f74441d None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:20.341366 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application [None req-edbe687a-e7c4-4b31-be4d-4158f9f0730d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 14:46:20.341366 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:46:20.341366 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:46:20.341366 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:46:20.341366 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:46:20.341366 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:46:20.341366 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:46:20.341366 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:46:20.341366 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:46:20.341366 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:46:20.341366 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:46:20.341366 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:46:20.341366 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 14:46:20.341366 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 14:46:20.341366 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 14:46:20.342525 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 14:46:20.342525 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:46:20.342525 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:46:20.342525 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:46:20.342525 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:46:20.342525 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:46:20.342525 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:46:20.342525 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:46:20.342525 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application with Lock( Sep 01 14:46:20.342525 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:46:20.342525 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:46:20.342525 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:46:20.342525 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:46:20.342525 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:46:20.342525 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:46:20.342525 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:46:20.342525 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:46:20.343649 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 14:46:20.343649 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 14:46:20.343649 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 14:46:20.343649 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 14:46:20.343649 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 14:46:20.343649 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 14:46:20.343649 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 14:46:20.343649 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Sep 01 14:46:20.345334 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 329/680] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Sep 1 14:46:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:46:20.346196 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbe687a-e7c4-4b31-be4d-4158f9f0730d None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:20.374976 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:20.462309 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbe687a-e7c4-4b31-be4d-4158f9f0730d None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:20.464211 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14234a8-b160-4828-9e53-5c985f74441d None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:20.500608 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382d7729-8d87-4da2-b115-04b306eb63cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:20.501032 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04184d28-bebe-4dde-81f0-187d3408fbfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:20.501232 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382d7729-8d87-4da2-b115-04b306eb63cd None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:20.501488 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382d7729-8d87-4da2-b115-04b306eb63cd None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:20.501655 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04184d28-bebe-4dde-81f0-187d3408fbfa None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:20.501841 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382d7729-8d87-4da2-b115-04b306eb63cd None admin] PATH_INFO: `/v3/domains` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:20.501920 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04184d28-bebe-4dde-81f0-187d3408fbfa None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:20.502145 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04184d28-bebe-4dde-81f0-187d3408fbfa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:20.502414 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382d7729-8d87-4da2-b115-04b306eb63cd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:46:20.502717 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382d7729-8d87-4da2-b115-04b306eb63cd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:20.502901 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04184d28-bebe-4dde-81f0-187d3408fbfa None admin] RBAC: Adding query filter params (role.id=5e3f992e804046f381b3de8bf2a512e2, scope.project.id=e8c85b4bd17e4e89b56bf0f831b474d5, user.id=2790149639aa418bbc4e4982e93eda44) {{(pid=81341) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:46:20.503204 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04184d28-bebe-4dde-81f0-187d3408fbfa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:20.504083 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382d7729-8d87-4da2-b115-04b306eb63cd None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:20.505162 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04184d28-bebe-4dde-81f0-187d3408fbfa None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:20.511550 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 330/681] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Sep 1 14:46:20 2022] GET /identity/v3/role_assignments?user.id=2790149639aa418bbc4e4982e93eda44&scope.project.id=e8c85b4bd17e4e89b56bf0f831b474d5&role.id=5e3f992e804046f381b3de8bf2a512e2 => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:46:20.512301 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04184d28-bebe-4dde-81f0-187d3408fbfa None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:20.529137 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 352/682] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Sep 1 14:46:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:46:20.530546 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382d7729-8d87-4da2-b115-04b306eb63cd None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:20.547923 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:20.644030 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04184d28-bebe-4dde-81f0-187d3408fbfa None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:20.664771 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:20.690765 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382d7729-8d87-4da2-b115-04b306eb63cd None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:20.697375 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb94687-4708-4fe8-ad61-c9962afa7e9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:20.697995 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb94687-4708-4fe8-ad61-c9962afa7e9f None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:20.700012 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb94687-4708-4fe8-ad61-c9962afa7e9f None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:20.700012 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb94687-4708-4fe8-ad61-c9962afa7e9f None admin] PATH_INFO: `/v3/projects` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:20.700012 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb94687-4708-4fe8-ad61-c9962afa7e9f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81341) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:46:20.700283 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb94687-4708-4fe8-ad61-c9962afa7e9f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:20.701886 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb94687-4708-4fe8-ad61-c9962afa7e9f None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:20.718800 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 331/683] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Sep 1 14:46:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:46:20.722274 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb94687-4708-4fe8-ad61-c9962afa7e9f None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:20.740108 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891d852a-a098-4977-9807-9f3c3c1e0a6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:20.740882 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891d852a-a098-4977-9807-9f3c3c1e0a6f None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:20.741528 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891d852a-a098-4977-9807-9f3c3c1e0a6f None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:20.741528 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891d852a-a098-4977-9807-9f3c3c1e0a6f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:20.755896 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891d852a-a098-4977-9807-9f3c3c1e0a6f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 14:46:20.760736 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application [None req-891d852a-a098-4977-9807-9f3c3c1e0a6f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 14:46:20.760736 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:46:20.760736 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:46:20.760736 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:46:20.760736 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:46:20.760736 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:46:20.760736 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:46:20.760736 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:46:20.760736 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:46:20.760736 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:46:20.760736 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:46:20.760736 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:46:20.760736 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 14:46:20.760736 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 14:46:20.760736 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 14:46:20.760736 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 14:46:20.761741 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 14:46:20.761741 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 14:46:20.761741 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 14:46:20.761741 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 14:46:20.761741 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:46:20.761741 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:46:20.761741 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:46:20.761741 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:46:20.761741 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:46:20.761741 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:46:20.761741 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:46:20.761741 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application with Lock( Sep 01 14:46:20.761741 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:46:20.761741 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:46:20.761741 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:46:20.761741 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:46:20.762673 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:46:20.762673 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:46:20.762673 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:46:20.762673 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:46:20.762673 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 14:46:20.762673 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 14:46:20.762673 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 14:46:20.762673 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 14:46:20.762673 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 14:46:20.762673 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 14:46:20.762673 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 14:46:20.762673 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Sep 01 14:46:20.762673 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 353/684] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Sep 1 14:46:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:46:20.763401 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891d852a-a098-4977-9807-9f3c3c1e0a6f None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:20.846612 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:20.854174 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb94687-4708-4fe8-ad61-c9962afa7e9f None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:20.874595 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891d852a-a098-4977-9807-9f3c3c1e0a6f None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:20.903812 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591908ed-d0e0-49f1-ac02-be80686f74c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:20.904445 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591908ed-d0e0-49f1-ac02-be80686f74c2 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:20.904724 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591908ed-d0e0-49f1-ac02-be80686f74c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:20.905245 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591908ed-d0e0-49f1-ac02-be80686f74c2 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:20.915036 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c29f28-460a-4e58-a1b8-92db864b7f26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:20.915223 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591908ed-d0e0-49f1-ac02-be80686f74c2 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:20.916151 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c29f28-460a-4e58-a1b8-92db864b7f26 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:20.916652 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c29f28-460a-4e58-a1b8-92db864b7f26 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:20.916827 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591908ed-d0e0-49f1-ac02-be80686f74c2 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:20.917463 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c29f28-460a-4e58-a1b8-92db864b7f26 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:20.918433 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c29f28-460a-4e58-a1b8-92db864b7f26 None admin] RBAC: Adding query filter params (role.id=5e3f992e804046f381b3de8bf2a512e2, scope.project.id=e8c85b4bd17e4e89b56bf0f831b474d5, user.id=d3b81739e34e4e648f88b6d73fcb5e46) {{(pid=81342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:46:20.918945 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c29f28-460a-4e58-a1b8-92db864b7f26 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:20.920591 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c29f28-460a-4e58-a1b8-92db864b7f26 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:20.927930 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 354/685] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Sep 1 14:46:20 2022] GET /identity/v3/role_assignments?user.id=d3b81739e34e4e648f88b6d73fcb5e46&scope.project.id=e8c85b4bd17e4e89b56bf0f831b474d5&role.id=5e3f992e804046f381b3de8bf2a512e2 => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:46:20.928997 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c29f28-460a-4e58-a1b8-92db864b7f26 None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:20.934838 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application [None req-591908ed-d0e0-49f1-ac02-be80686f74c2 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 01 14:46:20.934838 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:46:20.934838 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:46:20.934838 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:46:20.934838 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:46:20.934838 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:46:20.934838 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:46:20.934838 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:46:20.934838 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:46:20.934838 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:46:20.934838 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:46:20.934838 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:46:20.934838 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 14:46:20.934838 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 14:46:20.934838 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 14:46:20.934838 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 14:46:20.935875 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:46:20.935875 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:46:20.935875 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 14:46:20.935875 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 14:46:20.935875 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 14:46:20.935875 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 14:46:20.935875 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:46:20.935875 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:46:20.935875 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:46:20.935875 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:46:20.935875 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:46:20.935875 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application with Lock( Sep 01 14:46:20.935875 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:46:20.935875 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:46:20.935875 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:46:20.935875 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:46:20.935875 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:46:20.935875 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:46:20.936867 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:46:20.936867 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:46:20.936867 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 14:46:20.936867 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 14:46:20.936867 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 14:46:20.936867 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 14:46:20.936867 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 14:46:20.936867 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 14:46:20.936867 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 01 14:46:20.936867 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Sep 01 14:46:20.944599 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 332/686] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Sep 1 14:46:20 2022] GET /identity/v3/users/swift => generated 83 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:46:20.946470 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591908ed-d0e0-49f1-ac02-be80686f74c2 None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:20.960943 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:21.109490 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591908ed-d0e0-49f1-ac02-be80686f74c2 None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:21.150791 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5ab4d1-7a9d-4556-9d23-3ce7a44d3fc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:21.151443 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c29f28-460a-4e58-a1b8-92db864b7f26 None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:21.151538 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5ab4d1-7a9d-4556-9d23-3ce7a44d3fc9 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:21.151770 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5ab4d1-7a9d-4556-9d23-3ce7a44d3fc9 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:21.151969 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5ab4d1-7a9d-4556-9d23-3ce7a44d3fc9 None admin] PATH_INFO: `/v3/users` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:21.152982 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5ab4d1-7a9d-4556-9d23-3ce7a44d3fc9 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81341) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:46:21.153290 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5ab4d1-7a9d-4556-9d23-3ce7a44d3fc9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:21.155775 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5ab4d1-7a9d-4556-9d23-3ce7a44d3fc9 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:21.173444 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:21.180050 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 333/687] 10.222.0.33 () {56 vars in 1135 bytes} [Thu Sep 1 14:46:20 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 14:46:21.187543 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5ab4d1-7a9d-4556-9d23-3ce7a44d3fc9 None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:21.203831 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6b4dc2-df19-431d-a278-70626d1624ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:21.204588 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6b4dc2-df19-431d-a278-70626d1624ba None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:21.204942 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6b4dc2-df19-431d-a278-70626d1624ba None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:21.205450 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6b4dc2-df19-431d-a278-70626d1624ba None admin] PATH_INFO: `/v3/roles` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:21.206064 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6b4dc2-df19-431d-a278-70626d1624ba None admin] RBAC: Adding query filter params (name=admin) {{(pid=81342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:46:21.206460 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6b4dc2-df19-431d-a278-70626d1624ba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:21.208439 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6b4dc2-df19-431d-a278-70626d1624ba None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:21.218580 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 355/688] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Sep 1 14:46:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:46:21.225334 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6b4dc2-df19-431d-a278-70626d1624ba None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:21.316088 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5ab4d1-7a9d-4556-9d23-3ce7a44d3fc9 None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:21.321195 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:21.352852 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6b4dc2-df19-431d-a278-70626d1624ba None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:21.353071 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c168da42-cfa0-4f9c-a2a7-1428e35b9cd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:21.353582 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c168da42-cfa0-4f9c-a2a7-1428e35b9cd6 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:21.353859 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c168da42-cfa0-4f9c-a2a7-1428e35b9cd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:21.354124 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c168da42-cfa0-4f9c-a2a7-1428e35b9cd6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:21.392426 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646450a1-0a47-46d2-9c09-222eb32ed848 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:21.392976 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646450a1-0a47-46d2-9c09-222eb32ed848 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:21.393325 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646450a1-0a47-46d2-9c09-222eb32ed848 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:21.393595 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646450a1-0a47-46d2-9c09-222eb32ed848 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:21.406997 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646450a1-0a47-46d2-9c09-222eb32ed848 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:21.408466 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646450a1-0a47-46d2-9c09-222eb32ed848 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:21.424574 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application [None req-646450a1-0a47-46d2-9c09-222eb32ed848 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 14:46:21.424574 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:46:21.424574 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:46:21.424574 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:46:21.424574 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:46:21.424574 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:46:21.424574 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:46:21.424574 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:46:21.424574 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:46:21.424574 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:46:21.424574 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:46:21.424574 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:46:21.424574 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 14:46:21.424574 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 14:46:21.424574 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 14:46:21.424574 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 14:46:21.425587 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:46:21.425587 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:46:21.425587 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:46:21.425587 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:46:21.425587 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:46:21.425587 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:46:21.425587 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:46:21.425587 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application with Lock( Sep 01 14:46:21.425587 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:46:21.425587 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:46:21.425587 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:46:21.425587 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:46:21.425587 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:46:21.425587 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:46:21.425587 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:46:21.425587 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:46:21.425587 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 14:46:21.426634 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 14:46:21.426634 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 14:46:21.426634 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Sep 01 14:46:21.426634 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 356/689] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 1 14:46:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:46:21.427322 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e397c2ac-eeed-42ea-92c9-1329e85e0aac None None] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:21.427601 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e397c2ac-eeed-42ea-92c9-1329e85e0aac None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:21.427866 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e397c2ac-eeed-42ea-92c9-1329e85e0aac None None] PATH_INFO: `/` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:21.433260 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 357/690] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 1 14:46:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:46:21.433962 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c168da42-cfa0-4f9c-a2a7-1428e35b9cd6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:21.434164 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e397c2ac-eeed-42ea-92c9-1329e85e0aac None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:21.435519 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c168da42-cfa0-4f9c-a2a7-1428e35b9cd6 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:21.450595 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:21.545869 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 334/691] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Sep 1 14:46:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 14:46:21.547412 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df43805-aea8-41a7-b28b-fd45c51a23c6 None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:21.548178 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df43805-aea8-41a7-b28b-fd45c51a23c6 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:21.548456 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df43805-aea8-41a7-b28b-fd45c51a23c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:21.566081 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e397c2ac-eeed-42ea-92c9-1329e85e0aac None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:21.569986 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-3df43805-aea8-41a7-b28b-fd45c51a23c6 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:46:21.571672 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-3df43805-aea8-41a7-b28b-fd45c51a23c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:46:21.601458 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:21.610855 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee105c9-d4d3-459b-a397-98a5bef09028 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:21.611481 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee105c9-d4d3-459b-a397-98a5bef09028 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:21.611759 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee105c9-d4d3-459b-a397-98a5bef09028 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:21.612081 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee105c9-d4d3-459b-a397-98a5bef09028 None admin] PATH_INFO: `/v3/domains` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:21.612746 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee105c9-d4d3-459b-a397-98a5bef09028 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:46:21.613047 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee105c9-d4d3-459b-a397-98a5bef09028 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:21.614472 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee105c9-d4d3-459b-a397-98a5bef09028 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:21.630675 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 358/692] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Sep 1 14:46:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:46:21.631705 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee105c9-d4d3-459b-a397-98a5bef09028 None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:21.647839 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 335/693] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 1 14:46:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 14:46:21.648878 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df43805-aea8-41a7-b28b-fd45c51a23c6 None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:21.708477 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:21.753032 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df43805-aea8-41a7-b28b-fd45c51a23c6 None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:21.787539 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee105c9-d4d3-459b-a397-98a5bef09028 None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:21.807135 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788e1e3e-4679-4c9f-8dc6-b3d4e8533eed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:21.807772 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788e1e3e-4679-4c9f-8dc6-b3d4e8533eed None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:21.808045 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788e1e3e-4679-4c9f-8dc6-b3d4e8533eed None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:21.808309 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788e1e3e-4679-4c9f-8dc6-b3d4e8533eed None admin] PATH_INFO: `/v3/users/nova` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:21.820298 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788e1e3e-4679-4c9f-8dc6-b3d4e8533eed None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:21.822221 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788e1e3e-4679-4c9f-8dc6-b3d4e8533eed None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:21.845149 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application [None req-788e1e3e-4679-4c9f-8dc6-b3d4e8533eed None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 01 14:46:21.845149 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:46:21.845149 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:46:21.845149 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:46:21.845149 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:46:21.845149 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:46:21.845149 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:46:21.845149 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:46:21.845149 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:46:21.845149 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:46:21.845149 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:46:21.845149 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:46:21.845149 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 14:46:21.845149 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 14:46:21.845149 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 14:46:21.845149 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 14:46:21.846164 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:46:21.846164 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:46:21.846164 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 14:46:21.846164 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 14:46:21.846164 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 14:46:21.846164 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 14:46:21.846164 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:46:21.846164 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:46:21.846164 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:46:21.846164 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:46:21.846164 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:46:21.846164 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application with Lock( Sep 01 14:46:21.846164 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:46:21.846164 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:46:21.846164 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:46:21.846164 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:46:21.846164 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:46:21.846164 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:46:21.847716 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:46:21.847716 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:46:21.847716 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 14:46:21.847716 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 14:46:21.847716 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 14:46:21.847716 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 14:46:21.847716 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 14:46:21.847716 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 14:46:21.847716 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 01 14:46:21.847716 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Sep 01 14:46:21.847716 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 336/694] 10.222.0.33 () {56 vars in 1093 bytes} [Thu Sep 1 14:46:21 2022] GET /identity/v3/users/nova => generated 82 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:46:21.854480 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788e1e3e-4679-4c9f-8dc6-b3d4e8533eed None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:21.855941 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc9a047-6e9b-4ea5-aa46-5c35d52cb609 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:21.856565 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc9a047-6e9b-4ea5-aa46-5c35d52cb609 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:21.856838 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc9a047-6e9b-4ea5-aa46-5c35d52cb609 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:21.857151 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc9a047-6e9b-4ea5-aa46-5c35d52cb609 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:21.880340 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc9a047-6e9b-4ea5-aa46-5c35d52cb609 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:21.882624 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc9a047-6e9b-4ea5-aa46-5c35d52cb609 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:21.907853 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application [None req-8dc9a047-6e9b-4ea5-aa46-5c35d52cb609 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 14:46:21.907853 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:46:21.907853 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:46:21.907853 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:46:21.907853 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:46:21.907853 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:46:21.907853 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:46:21.907853 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:46:21.907853 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:46:21.907853 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:46:21.907853 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:46:21.907853 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:46:21.907853 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 14:46:21.907853 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 14:46:21.907853 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 14:46:21.907853 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 14:46:21.910004 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:46:21.910004 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:46:21.910004 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:46:21.910004 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:46:21.910004 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:46:21.910004 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:46:21.910004 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:46:21.910004 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application with Lock( Sep 01 14:46:21.910004 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:46:21.910004 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:46:21.910004 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:46:21.910004 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:46:21.910004 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:46:21.910004 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:46:21.910004 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:46:21.910004 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:46:21.910004 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 14:46:21.912205 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 14:46:21.912205 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 14:46:21.912205 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Sep 01 14:46:21.918171 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 359/695] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 1 14:46:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:46:21.919129 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc9a047-6e9b-4ea5-aa46-5c35d52cb609 None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:21.970764 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:22.036332 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc9a047-6e9b-4ea5-aa46-5c35d52cb609 None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:22.041534 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:22.055795 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788e1e3e-4679-4c9f-8dc6-b3d4e8533eed None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:22.074844 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2414792d-0321-4a46-abc9-6346738fc3b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:22.075458 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2414792d-0321-4a46-abc9-6346738fc3b8 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:22.075785 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2414792d-0321-4a46-abc9-6346738fc3b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:22.076070 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2414792d-0321-4a46-abc9-6346738fc3b8 None admin] PATH_INFO: `/v3/domains` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:22.076721 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2414792d-0321-4a46-abc9-6346738fc3b8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:46:22.077026 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2414792d-0321-4a46-abc9-6346738fc3b8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:22.078429 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2414792d-0321-4a46-abc9-6346738fc3b8 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:22.090468 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 360/696] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Sep 1 14:46:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:46:22.091082 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2414792d-0321-4a46-abc9-6346738fc3b8 None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:22.101964 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfa74c7-2f68-42e7-86b7-2cf1cdb1e05d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:22.102729 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfa74c7-2f68-42e7-86b7-2cf1cdb1e05d None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:22.103015 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfa74c7-2f68-42e7-86b7-2cf1cdb1e05d None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:22.103286 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfa74c7-2f68-42e7-86b7-2cf1cdb1e05d None admin] PATH_INFO: `/v3/users` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:22.104026 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfa74c7-2f68-42e7-86b7-2cf1cdb1e05d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81341) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:46:22.104329 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfa74c7-2f68-42e7-86b7-2cf1cdb1e05d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:22.105985 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfa74c7-2f68-42e7-86b7-2cf1cdb1e05d None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:22.125801 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 337/697] 10.222.0.33 () {56 vars in 1133 bytes} [Thu Sep 1 14:46:21 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:46:22.130843 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfa74c7-2f68-42e7-86b7-2cf1cdb1e05d None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:22.210670 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:22.218509 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2414792d-0321-4a46-abc9-6346738fc3b8 None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:22.250355 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfa74c7-2f68-42e7-86b7-2cf1cdb1e05d None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:22.256104 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3baa1d8e-cb0d-4f24-b5a7-bb2f1317ff02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:22.256778 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3baa1d8e-cb0d-4f24-b5a7-bb2f1317ff02 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:22.257058 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3baa1d8e-cb0d-4f24-b5a7-bb2f1317ff02 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:22.257326 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3baa1d8e-cb0d-4f24-b5a7-bb2f1317ff02 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:22.268113 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3baa1d8e-cb0d-4f24-b5a7-bb2f1317ff02 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:22.269562 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3baa1d8e-cb0d-4f24-b5a7-bb2f1317ff02 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:22.284564 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application [None req-3baa1d8e-cb0d-4f24-b5a7-bb2f1317ff02 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 14:46:22.284564 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:46:22.284564 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:46:22.284564 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:46:22.284564 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:46:22.284564 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:46:22.284564 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:46:22.284564 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:46:22.284564 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:46:22.284564 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:46:22.284564 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:46:22.284564 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:46:22.284564 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 14:46:22.284564 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 14:46:22.284564 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 14:46:22.284564 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 14:46:22.286782 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:46:22.286782 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:46:22.286782 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:46:22.286782 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:46:22.286782 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:46:22.286782 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:46:22.286782 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:46:22.286782 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application with Lock( Sep 01 14:46:22.286782 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:46:22.286782 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:46:22.286782 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:46:22.286782 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:46:22.286782 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:46:22.286782 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:46:22.286782 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:46:22.286782 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:46:22.286782 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 14:46:22.287813 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 14:46:22.287813 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 14:46:22.287813 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Sep 01 14:46:22.287813 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 361/698] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 1 14:46:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:46:22.287813 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3baa1d8e-cb0d-4f24-b5a7-bb2f1317ff02 None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:22.295881 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428ffb6a-800a-4afd-a6ff-00ada5d99402 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:22.296541 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428ffb6a-800a-4afd-a6ff-00ada5d99402 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:22.296814 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428ffb6a-800a-4afd-a6ff-00ada5d99402 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:22.297081 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428ffb6a-800a-4afd-a6ff-00ada5d99402 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:22.306762 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428ffb6a-800a-4afd-a6ff-00ada5d99402 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:22.309052 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428ffb6a-800a-4afd-a6ff-00ada5d99402 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:22.323046 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application [None req-428ffb6a-800a-4afd-a6ff-00ada5d99402 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 14:46:22.323046 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:46:22.323046 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:46:22.323046 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:46:22.323046 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:46:22.323046 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:46:22.323046 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:46:22.323046 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:46:22.323046 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:46:22.323046 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:46:22.323046 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:46:22.323046 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:46:22.323046 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 14:46:22.323046 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 14:46:22.323046 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 14:46:22.324050 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 14:46:22.324050 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:46:22.324050 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:46:22.324050 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:46:22.324050 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:46:22.324050 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:46:22.324050 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:46:22.324050 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:46:22.324050 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application with Lock( Sep 01 14:46:22.324050 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:46:22.324050 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:46:22.324050 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:46:22.324050 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:46:22.324050 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:46:22.324050 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:46:22.324050 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:46:22.324050 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:46:22.325043 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 14:46:22.325043 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 14:46:22.325043 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 14:46:22.325043 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 14:46:22.325043 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 14:46:22.325043 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 14:46:22.325043 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 14:46:22.325043 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Sep 01 14:46:22.327547 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 338/699] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Sep 1 14:46:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:46:22.328695 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428ffb6a-800a-4afd-a6ff-00ada5d99402 None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:22.341303 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:22.416919 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3baa1d8e-cb0d-4f24-b5a7-bb2f1317ff02 None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:22.435206 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428ffb6a-800a-4afd-a6ff-00ada5d99402 None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:22.453419 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f8e08c-534d-4035-b69a-9b33c945573a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:22.454267 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f8e08c-534d-4035-b69a-9b33c945573a None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:22.454455 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f8e08c-534d-4035-b69a-9b33c945573a None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:22.454685 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f8e08c-534d-4035-b69a-9b33c945573a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:22.472379 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fccb31-39e5-4cbd-b44c-af8c119cfb8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:22.473951 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fccb31-39e5-4cbd-b44c-af8c119cfb8b None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:22.474294 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:22.475022 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fccb31-39e5-4cbd-b44c-af8c119cfb8b None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:22.475512 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fccb31-39e5-4cbd-b44c-af8c119cfb8b None admin] PATH_INFO: `/v3/domains` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:22.476549 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fccb31-39e5-4cbd-b44c-af8c119cfb8b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81341) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:46:22.477068 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fccb31-39e5-4cbd-b44c-af8c119cfb8b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:22.478708 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fccb31-39e5-4cbd-b44c-af8c119cfb8b None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:22.496629 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 339/700] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Sep 1 14:46:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:46:22.497889 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fccb31-39e5-4cbd-b44c-af8c119cfb8b None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:22.527103 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f8e08c-534d-4035-b69a-9b33c945573a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:22.528697 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f8e08c-534d-4035-b69a-9b33c945573a None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:22.571088 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:22.609049 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fccb31-39e5-4cbd-b44c-af8c119cfb8b None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:22.635849 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 362/701] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Sep 1 14:46:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 14:46:22.636920 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f8e08c-534d-4035-b69a-9b33c945573a None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:22.651049 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201a1ed9-fc24-432c-889a-127e62ee6cc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:22.651813 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201a1ed9-fc24-432c-889a-127e62ee6cc5 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:22.652197 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201a1ed9-fc24-432c-889a-127e62ee6cc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:22.652568 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201a1ed9-fc24-432c-889a-127e62ee6cc5 None admin] PATH_INFO: `/v3/projects` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:22.653474 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201a1ed9-fc24-432c-889a-127e62ee6cc5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81341) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:46:22.653885 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201a1ed9-fc24-432c-889a-127e62ee6cc5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:22.655651 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201a1ed9-fc24-432c-889a-127e62ee6cc5 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:22.669802 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 340/702] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Sep 1 14:46:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:46:22.670504 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201a1ed9-fc24-432c-889a-127e62ee6cc5 None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:22.691815 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:22.747116 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f8e08c-534d-4035-b69a-9b33c945573a None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:22.777299 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201a1ed9-fc24-432c-889a-127e62ee6cc5 None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:22.782348 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf0522e-c0fd-4792-9369-b6889e735cde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:22.783197 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf0522e-c0fd-4792-9369-b6889e735cde None admin] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:22.783682 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf0522e-c0fd-4792-9369-b6889e735cde None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:22.784178 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf0522e-c0fd-4792-9369-b6889e735cde None admin] PATH_INFO: `/v3/users` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:22.785090 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf0522e-c0fd-4792-9369-b6889e735cde None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:22.787444 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf0522e-c0fd-4792-9369-b6889e735cde None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:22.803874 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:22.818283 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 363/703] 10.222.0.33 () {60 vars in 1132 bytes} [Thu Sep 1 14:46:22 2022] POST /identity/v3/users => generated 254 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:46:22.818283 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd739b8-f49c-4117-a865-1991ffd2d7a4 None None] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:22.818283 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd739b8-f49c-4117-a865-1991ffd2d7a4 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:22.818283 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd739b8-f49c-4117-a865-1991ffd2d7a4 None None] PATH_INFO: `/` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:22.818728 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e0ddb9-8ff6-4aa2-a804-d858b4a86aad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:22.818728 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e0ddb9-8ff6-4aa2-a804-d858b4a86aad None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:22.818728 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e0ddb9-8ff6-4aa2-a804-d858b4a86aad None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:22.818728 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e0ddb9-8ff6-4aa2-a804-d858b4a86aad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:22.819027 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 364/704] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 1 14:46:22 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:46:22.820092 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd739b8-f49c-4117-a865-1991ffd2d7a4 None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:22.830577 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e0ddb9-8ff6-4aa2-a804-d858b4a86aad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:22.832176 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e0ddb9-8ff6-4aa2-a804-d858b4a86aad None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:22.851063 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application [None req-a6e0ddb9-8ff6-4aa2-a804-d858b4a86aad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 14:46:22.851063 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:46:22.851063 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:46:22.851063 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:46:22.851063 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:46:22.851063 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:46:22.851063 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:46:22.851063 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:46:22.851063 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:46:22.851063 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:46:22.851063 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:46:22.851063 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:46:22.851063 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 14:46:22.851063 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 14:46:22.851063 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 14:46:22.851063 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 14:46:22.852197 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:46:22.852197 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:46:22.852197 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:46:22.852197 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:46:22.852197 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:46:22.852197 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:46:22.852197 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:46:22.852197 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application with Lock( Sep 01 14:46:22.852197 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:46:22.852197 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:46:22.852197 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:46:22.852197 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:46:22.852197 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:46:22.852197 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:46:22.852197 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:46:22.852197 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:46:22.852197 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 14:46:22.853129 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 14:46:22.853129 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 14:46:22.853129 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Sep 01 14:46:22.857201 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 341/705] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 1 14:46:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:46:22.858713 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c224594a-7229-4348-bf7e-a17fd7473cbc None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:22.858997 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c224594a-7229-4348-bf7e-a17fd7473cbc None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:22.859274 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c224594a-7229-4348-bf7e-a17fd7473cbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:22.884417 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-c224594a-7229-4348-bf7e-a17fd7473cbc None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:46:22.886170 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-c224594a-7229-4348-bf7e-a17fd7473cbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:46:22.961761 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd739b8-f49c-4117-a865-1991ffd2d7a4 None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:22.965904 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:22.990805 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 342/706] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 1 14:46:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 14:46:22.991665 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c224594a-7229-4348-bf7e-a17fd7473cbc None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:23.009258 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc1f4fa-9db1-4022-bdf2-a4ae0c4332ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:23.010205 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc1f4fa-9db1-4022-bdf2-a4ae0c4332ad None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:23.010964 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc1f4fa-9db1-4022-bdf2-a4ae0c4332ad None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:23.011718 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc1f4fa-9db1-4022-bdf2-a4ae0c4332ad None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:23.013995 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc1f4fa-9db1-4022-bdf2-a4ae0c4332ad None admin] RBAC: Adding query filter params (role.id=5e3f992e804046f381b3de8bf2a512e2, scope.project.id=e8c85b4bd17e4e89b56bf0f831b474d5, user.id=73a5bda5a6ea46e9863ccafd57dee339) {{(pid=81342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:46:23.015152 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc1f4fa-9db1-4022-bdf2-a4ae0c4332ad None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:23.018486 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc1f4fa-9db1-4022-bdf2-a4ae0c4332ad None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:23.027480 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 365/707] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Sep 1 14:46:22 2022] GET /identity/v3/role_assignments?user.id=73a5bda5a6ea46e9863ccafd57dee339&scope.project.id=e8c85b4bd17e4e89b56bf0f831b474d5&role.id=5e3f992e804046f381b3de8bf2a512e2 => generated 608 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:46:23.114624 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c224594a-7229-4348-bf7e-a17fd7473cbc None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:23.132142 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:23.159033 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b053c8cd-bcb1-43f2-8161-29a1cca27171 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:23.160023 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b053c8cd-bcb1-43f2-8161-29a1cca27171 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:23.160535 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b053c8cd-bcb1-43f2-8161-29a1cca27171 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:23.161062 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b053c8cd-bcb1-43f2-8161-29a1cca27171 None admin] PATH_INFO: `/v3/domains` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:23.161952 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b053c8cd-bcb1-43f2-8161-29a1cca27171 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81341) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:46:23.162529 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b053c8cd-bcb1-43f2-8161-29a1cca27171 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:23.164513 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b053c8cd-bcb1-43f2-8161-29a1cca27171 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:23.165541 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2bdb8f-10bb-4fb7-bc26-60d8b2098c12 None None] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:23.165824 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2bdb8f-10bb-4fb7-bc26-60d8b2098c12 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:23.166092 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2bdb8f-10bb-4fb7-bc26-60d8b2098c12 None None] PATH_INFO: `/` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:23.168860 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 366/708] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 1 14:46:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 14:46:23.173351 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96568f47-2399-4cc4-9c09-eb76f9023f17 None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:23.173628 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96568f47-2399-4cc4-9c09-eb76f9023f17 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:23.173900 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96568f47-2399-4cc4-9c09-eb76f9023f17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:23.180293 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 343/709] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Sep 1 14:46:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:46:23.186347 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b053c8cd-bcb1-43f2-8161-29a1cca27171 None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:23.197213 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-96568f47-2399-4cc4-9c09-eb76f9023f17 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:46:23.198796 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-96568f47-2399-4cc4-9c09-eb76f9023f17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:46:23.298350 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 367/710] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 1 14:46:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 14:46:23.305246 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b053c8cd-bcb1-43f2-8161-29a1cca27171 None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:23.336513 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96568f47-2399-4cc4-9c09-eb76f9023f17 None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:23.345679 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f86c0a-5730-48cc-87b8-231ec769af23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:23.346757 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f86c0a-5730-48cc-87b8-231ec769af23 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:23.347247 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f86c0a-5730-48cc-87b8-231ec769af23 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:23.347776 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f86c0a-5730-48cc-87b8-231ec769af23 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:23.360589 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f86c0a-5730-48cc-87b8-231ec769af23 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:23.363371 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f86c0a-5730-48cc-87b8-231ec769af23 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:23.383686 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application [None req-24f86c0a-5730-48cc-87b8-231ec769af23 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 14:46:23.383686 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:46:23.383686 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:46:23.383686 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:46:23.383686 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:46:23.383686 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:46:23.383686 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:46:23.383686 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:46:23.383686 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:46:23.383686 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:46:23.383686 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:46:23.383686 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:46:23.383686 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 14:46:23.383686 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 14:46:23.383686 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 14:46:23.384639 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 14:46:23.384639 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:46:23.384639 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:46:23.384639 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:46:23.384639 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:46:23.384639 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:46:23.384639 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:46:23.384639 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:46:23.384639 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application with Lock( Sep 01 14:46:23.384639 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:46:23.384639 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:46:23.384639 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:46:23.384639 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:46:23.384639 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:46:23.384639 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:46:23.384639 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:46:23.384639 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:46:23.385571 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 14:46:23.385571 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 14:46:23.385571 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 14:46:23.385571 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 14:46:23.385571 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 14:46:23.385571 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 14:46:23.385571 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 14:46:23.385571 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Sep 01 14:46:23.385571 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 344/711] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Sep 1 14:46:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 14:46:23.390288 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f86c0a-5730-48cc-87b8-231ec769af23 None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:23.432215 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:23.486801 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:23.494780 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96568f47-2399-4cc4-9c09-eb76f9023f17 None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:23.533981 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f86c0a-5730-48cc-87b8-231ec769af23 None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:23.557539 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfd8f92-3dbf-4932-b68b-295fe7140071 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:23.558072 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfd8f92-3dbf-4932-b68b-295fe7140071 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:23.558368 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfd8f92-3dbf-4932-b68b-295fe7140071 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:23.558633 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfd8f92-3dbf-4932-b68b-295fe7140071 None admin] PATH_INFO: `/v3/services/network` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:23.559211 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfd8f92-3dbf-4932-b68b-295fe7140071 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:23.560565 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfd8f92-3dbf-4932-b68b-295fe7140071 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:23.583782 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application [None req-8bfd8f92-3dbf-4932-b68b-295fe7140071 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 01 14:46:23.583782 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:46:23.583782 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 14:46:23.583782 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 14:46:23.583782 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 14:46:23.583782 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 14:46:23.583782 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 14:46:23.583782 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 14:46:23.583782 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 01 14:46:23.583782 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Sep 01 14:46:23.583782 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 14:46:23.583782 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Sep 01 14:46:23.583782 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:46:23.583782 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:46:23.583782 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:46:23.583782 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:46:23.583782 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:46:23.584927 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:46:23.584927 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:46:23.584927 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:46:23.584927 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:46:23.584927 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:46:23.584927 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:46:23.584927 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 14:46:23.584927 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 14:46:23.584927 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 14:46:23.584927 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 14:46:23.584927 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:46:23.584927 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:46:23.584927 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:46:23.584927 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:46:23.584927 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:46:23.584927 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:46:23.585902 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:46:23.585902 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application with Lock( Sep 01 14:46:23.585902 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:46:23.585902 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:46:23.585902 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:46:23.585902 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:46:23.585902 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:46:23.585902 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:46:23.585902 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:46:23.585902 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:46:23.585902 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 14:46:23.585902 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 14:46:23.585902 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 01 14:46:23.585902 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Sep 01 14:46:23.586784 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5d6dda-e39a-4830-ba43-9d819f7c57a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:23.586784 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5d6dda-e39a-4830-ba43-9d819f7c57a5 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:23.586784 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5d6dda-e39a-4830-ba43-9d819f7c57a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:23.587012 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 368/712] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Sep 1 14:46:23 2022] GET /identity/v3/services/network => generated 88 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 14:46:23.587084 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5d6dda-e39a-4830-ba43-9d819f7c57a5 None admin] PATH_INFO: `/v3/projects` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:23.587153 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfd8f92-3dbf-4932-b68b-295fe7140071 None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:23.587221 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5d6dda-e39a-4830-ba43-9d819f7c57a5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81341) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:46:23.590990 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5d6dda-e39a-4830-ba43-9d819f7c57a5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:23.592288 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5d6dda-e39a-4830-ba43-9d819f7c57a5 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:23.606368 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 345/713] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Sep 1 14:46:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 14:46:23.607202 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5d6dda-e39a-4830-ba43-9d819f7c57a5 None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:23.693376 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfd8f92-3dbf-4932-b68b-295fe7140071 None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:23.729558 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:23.729852 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc4dced-3873-428f-ad9e-6c491ecf85e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:23.730565 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc4dced-3873-428f-ad9e-6c491ecf85e0 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:23.730880 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc4dced-3873-428f-ad9e-6c491ecf85e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:23.731089 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc4dced-3873-428f-ad9e-6c491ecf85e0 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:23.731716 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc4dced-3873-428f-ad9e-6c491ecf85e0 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:23.733088 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc4dced-3873-428f-ad9e-6c491ecf85e0 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:23.743079 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5d6dda-e39a-4830-ba43-9d819f7c57a5 None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:23.743529 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application [None req-9fc4dced-3873-428f-ad9e-6c491ecf85e0 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 01 14:46:23.743529 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:46:23.743529 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 14:46:23.743529 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 14:46:23.743529 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 14:46:23.743529 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 14:46:23.743529 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 14:46:23.743529 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 14:46:23.743529 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 01 14:46:23.743529 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Sep 01 14:46:23.743529 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 14:46:23.743529 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Sep 01 14:46:23.743529 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:46:23.743529 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:46:23.743529 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:46:23.743529 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:46:23.743529 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:46:23.744805 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:46:23.744805 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:46:23.744805 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:46:23.744805 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:46:23.744805 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:46:23.744805 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:46:23.744805 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 14:46:23.744805 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 14:46:23.744805 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 14:46:23.744805 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 14:46:23.744805 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:46:23.744805 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:46:23.744805 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:46:23.744805 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:46:23.744805 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:46:23.744805 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:46:23.745915 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:46:23.745915 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application with Lock( Sep 01 14:46:23.745915 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:46:23.745915 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:46:23.745915 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:46:23.745915 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:46:23.745915 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:46:23.745915 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:46:23.745915 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:46:23.745915 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:46:23.745915 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 14:46:23.745915 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 14:46:23.745915 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 01 14:46:23.745915 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Sep 01 14:46:23.745915 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 369/714] 10.222.0.33 () {56 vars in 1129 bytes} [Thu Sep 1 14:46:23 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:46:23.746777 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc4dced-3873-428f-ad9e-6c491ecf85e0 None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:23.781313 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:23.803147 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d45046a-e4f7-4f6b-b628-1b5baa5dd273 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:23.803705 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d45046a-e4f7-4f6b-b628-1b5baa5dd273 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:23.803988 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d45046a-e4f7-4f6b-b628-1b5baa5dd273 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:23.804257 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d45046a-e4f7-4f6b-b628-1b5baa5dd273 None admin] PATH_INFO: `/v3/services` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:23.804888 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d45046a-e4f7-4f6b-b628-1b5baa5dd273 None admin] RBAC: Adding query filter params (name=network) {{(pid=81341) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:46:23.805184 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d45046a-e4f7-4f6b-b628-1b5baa5dd273 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:23.806876 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d45046a-e4f7-4f6b-b628-1b5baa5dd273 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:23.816630 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 346/715] 10.222.0.33 () {56 vars in 1112 bytes} [Thu Sep 1 14:46:23 2022] GET /identity/v3/services?name=network => generated 124 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:46:23.817567 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d45046a-e4f7-4f6b-b628-1b5baa5dd273 None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:23.875713 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc4dced-3873-428f-ad9e-6c491ecf85e0 None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:23.921981 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d07b400-c8f5-4bc4-a126-a3aa94da8a4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:23.923467 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d07b400-c8f5-4bc4-a126-a3aa94da8a4c None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:23.923919 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d07b400-c8f5-4bc4-a126-a3aa94da8a4c None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:23.924567 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d07b400-c8f5-4bc4-a126-a3aa94da8a4c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:23.925863 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d07b400-c8f5-4bc4-a126-a3aa94da8a4c None admin] RBAC: Adding query filter params (role.id=2a2adb769e55476b82e5014897e0a9c0, scope.project.id=e8c85b4bd17e4e89b56bf0f831b474d5, user.id=4858bb8621854ec8aba836e7d9050498) {{(pid=81342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:46:23.927138 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d07b400-c8f5-4bc4-a126-a3aa94da8a4c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:23.928706 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d07b400-c8f5-4bc4-a126-a3aa94da8a4c None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:23.928886 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d45046a-e4f7-4f6b-b628-1b5baa5dd273 None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:23.935886 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 370/716] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Sep 1 14:46:23 2022] GET /identity/v3/role_assignments?user.id=4858bb8621854ec8aba836e7d9050498&scope.project.id=e8c85b4bd17e4e89b56bf0f831b474d5&role.id=2a2adb769e55476b82e5014897e0a9c0 => generated 259 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:46:23.936927 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d07b400-c8f5-4bc4-a126-a3aa94da8a4c None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:23.969966 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c8ec93-4b6a-4dc1-90c7-70557e53c0b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:23.970628 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c8ec93-4b6a-4dc1-90c7-70557e53c0b7 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:23.970901 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c8ec93-4b6a-4dc1-90c7-70557e53c0b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:23.971235 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c8ec93-4b6a-4dc1-90c7-70557e53c0b7 None admin] PATH_INFO: `/v3/services` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:23.971792 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c8ec93-4b6a-4dc1-90c7-70557e53c0b7 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81341) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:46:23.972087 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c8ec93-4b6a-4dc1-90c7-70557e53c0b7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:23.974281 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c8ec93-4b6a-4dc1-90c7-70557e53c0b7 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:23.979911 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 347/717] 10.222.0.33 () {56 vars in 1124 bytes} [Thu Sep 1 14:46:23 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:46:23.985821 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c8ec93-4b6a-4dc1-90c7-70557e53c0b7 None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:24.009277 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:24.091371 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:24.096740 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d07b400-c8f5-4bc4-a126-a3aa94da8a4c None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:24.151037 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4900214-549c-4d04-8cf5-6485f54dea77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:24.152277 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4900214-549c-4d04-8cf5-6485f54dea77 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:24.153350 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4900214-549c-4d04-8cf5-6485f54dea77 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:24.153831 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4900214-549c-4d04-8cf5-6485f54dea77 None admin] PATH_INFO: `/v3/services` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:24.155008 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4900214-549c-4d04-8cf5-6485f54dea77 None admin] RBAC: Adding query filter params (type=network) {{(pid=81342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:46:24.155514 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4900214-549c-4d04-8cf5-6485f54dea77 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:24.157667 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4900214-549c-4d04-8cf5-6485f54dea77 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:24.168734 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 371/718] 10.222.0.33 () {56 vars in 1112 bytes} [Thu Sep 1 14:46:23 2022] GET /identity/v3/services?type=network => generated 124 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:46:24.183020 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c8ec93-4b6a-4dc1-90c7-70557e53c0b7 None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:24.242325 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c5b8e3-1e14-49a7-90b0-76a1c7323197 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:24.242938 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c5b8e3-1e14-49a7-90b0-76a1c7323197 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:24.243213 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c5b8e3-1e14-49a7-90b0-76a1c7323197 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:24.243480 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c5b8e3-1e14-49a7-90b0-76a1c7323197 None admin] PATH_INFO: `/v3/services` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:24.244104 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c5b8e3-1e14-49a7-90b0-76a1c7323197 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81341) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:46:24.244398 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c5b8e3-1e14-49a7-90b0-76a1c7323197 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:24.245778 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c5b8e3-1e14-49a7-90b0-76a1c7323197 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:24.254691 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 348/719] 10.222.0.33 () {56 vars in 1124 bytes} [Thu Sep 1 14:46:23 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:46:25.179289 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6fa794-fa80-4f49-9428-7086a6d82de9 None None] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:25.180347 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6fa794-fa80-4f49-9428-7086a6d82de9 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:25.180847 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6fa794-fa80-4f49-9428-7086a6d82de9 None None] PATH_INFO: `/` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:25.182909 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 372/720] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 1 14:46:25 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:46:25.187806 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7604ce1e-ff46-4134-8833-487f33d001d6 None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:25.188618 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7604ce1e-ff46-4134-8833-487f33d001d6 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:25.189102 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7604ce1e-ff46-4134-8833-487f33d001d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:25.207974 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-7604ce1e-ff46-4134-8833-487f33d001d6 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:46:25.209473 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-7604ce1e-ff46-4134-8833-487f33d001d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:46:25.280403 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 349/721] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 1 14:46:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 14:46:25.564626 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d87d693-7c03-44cc-a84a-84838761dc7d None None] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:25.565001 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d87d693-7c03-44cc-a84a-84838761dc7d None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:25.565196 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d87d693-7c03-44cc-a84a-84838761dc7d None None] PATH_INFO: `/` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:25.566843 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 373/722] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 1 14:46:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:46:25.572191 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84eafbc-93c6-405d-95f2-de5176862256 None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:25.572478 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84eafbc-93c6-405d-95f2-de5176862256 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:25.572757 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84eafbc-93c6-405d-95f2-de5176862256 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:25.599668 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-a84eafbc-93c6-405d-95f2-de5176862256 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:46:25.600882 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-a84eafbc-93c6-405d-95f2-de5176862256 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:46:25.609508 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:25.666446 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d87d693-7c03-44cc-a84a-84838761dc7d None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:25.684833 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 350/723] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 1 14:46:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 14:46:25.825122 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d87d693-7c03-44cc-a84a-84838761dc7d None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:25.859166 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6072399b-15f0-4fd6-965c-d5574a36e340 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:25.859790 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6072399b-15f0-4fd6-965c-d5574a36e340 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:25.860064 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6072399b-15f0-4fd6-965c-d5574a36e340 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:25.860326 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6072399b-15f0-4fd6-965c-d5574a36e340 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:25.870473 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6072399b-15f0-4fd6-965c-d5574a36e340 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 14:46:25.875195 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application [None req-6072399b-15f0-4fd6-965c-d5574a36e340 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 14:46:25.875195 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:46:25.875195 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:46:25.875195 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:46:25.875195 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:46:25.875195 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:46:25.875195 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:46:25.875195 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:46:25.875195 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:46:25.875195 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:46:25.875195 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:46:25.875195 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:46:25.875195 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 14:46:25.875195 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 14:46:25.875195 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 14:46:25.875195 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 14:46:25.876146 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 14:46:25.876146 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 14:46:25.876146 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 14:46:25.876146 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 14:46:25.876146 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:46:25.876146 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:46:25.876146 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:46:25.876146 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:46:25.876146 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:46:25.876146 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:46:25.876146 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:46:25.876146 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application with Lock( Sep 01 14:46:25.876146 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:46:25.876146 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:46:25.876146 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:46:25.876146 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:46:25.877001 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:46:25.877001 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:46:25.877001 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:46:25.877001 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:46:25.877001 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 14:46:25.877001 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 14:46:25.877001 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 14:46:25.877001 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 14:46:25.877001 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 14:46:25.877001 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 14:46:25.877001 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 14:46:25.877001 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Sep 01 14:46:25.877001 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 374/724] 10.222.0.33 () {56 vars in 1102 bytes} [Thu Sep 1 14:46:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:46:25.882580 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84eafbc-93c6-405d-95f2-de5176862256 None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:25.986997 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6072399b-15f0-4fd6-965c-d5574a36e340 None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:26.001900 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84eafbc-93c6-405d-95f2-de5176862256 None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:26.024018 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:26.045746 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9565ace-85f0-456c-b6b1-c57641729f04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:26.046377 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9565ace-85f0-456c-b6b1-c57641729f04 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:26.046654 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9565ace-85f0-456c-b6b1-c57641729f04 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:26.046921 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9565ace-85f0-456c-b6b1-c57641729f04 None admin] PATH_INFO: `/v3/roles` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:26.047581 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9565ace-85f0-456c-b6b1-c57641729f04 None admin] RBAC: Adding query filter params (name=service) {{(pid=81341) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:46:26.047879 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9565ace-85f0-456c-b6b1-c57641729f04 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:26.049255 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9565ace-85f0-456c-b6b1-c57641729f04 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:26.058539 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 351/725] 10.222.0.33 () {56 vars in 1103 bytes} [Thu Sep 1 14:46:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 14:46:26.066428 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9565ace-85f0-456c-b6b1-c57641729f04 None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:26.078585 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:26.193146 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6072399b-15f0-4fd6-965c-d5574a36e340 None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:26.252514 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9565ace-85f0-456c-b6b1-c57641729f04 None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:26.255128 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca795df6-8c24-4e6b-af70-0a096bd53ebd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:26.255767 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca795df6-8c24-4e6b-af70-0a096bd53ebd None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:26.256043 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca795df6-8c24-4e6b-af70-0a096bd53ebd None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:26.256311 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca795df6-8c24-4e6b-af70-0a096bd53ebd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:26.268568 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca795df6-8c24-4e6b-af70-0a096bd53ebd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 14:46:26.273484 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application [None req-ca795df6-8c24-4e6b-af70-0a096bd53ebd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 14:46:26.273484 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:46:26.273484 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:46:26.273484 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:46:26.273484 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:46:26.273484 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:46:26.273484 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:46:26.273484 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:46:26.273484 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:46:26.273484 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:46:26.273484 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:46:26.273484 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:46:26.273484 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 14:46:26.273484 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 14:46:26.273484 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 14:46:26.273484 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 14:46:26.274462 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 14:46:26.274462 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 14:46:26.274462 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 14:46:26.274462 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 14:46:26.274462 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:46:26.274462 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:46:26.274462 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:46:26.274462 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:46:26.274462 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:46:26.274462 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:46:26.274462 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:46:26.274462 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application with Lock( Sep 01 14:46:26.274462 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:46:26.274462 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:46:26.274462 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:46:26.274462 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:46:26.275344 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:46:26.275344 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:46:26.275344 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:46:26.275344 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:46:26.275344 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 14:46:26.275344 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 14:46:26.275344 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 14:46:26.275344 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 14:46:26.275344 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 14:46:26.275344 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 14:46:26.275344 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 14:46:26.275344 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Sep 01 14:46:26.275344 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 375/726] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Sep 1 14:46:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 14:46:26.276573 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e301b16b-7982-49f0-9f15-a7f615b6d373 None None] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:26.276573 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e301b16b-7982-49f0-9f15-a7f615b6d373 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:26.276890 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e301b16b-7982-49f0-9f15-a7f615b6d373 None None] PATH_INFO: `/` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:26.282124 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 376/727] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 1 14:46:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:46:26.282124 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e301b16b-7982-49f0-9f15-a7f615b6d373 None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:26.301906 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32562bdf-c9b4-409f-8bb8-8220b54725e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:26.302569 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32562bdf-c9b4-409f-8bb8-8220b54725e5 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:26.302846 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32562bdf-c9b4-409f-8bb8-8220b54725e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:26.303113 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32562bdf-c9b4-409f-8bb8-8220b54725e5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:26.315808 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32562bdf-c9b4-409f-8bb8-8220b54725e5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:26.317766 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32562bdf-c9b4-409f-8bb8-8220b54725e5 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:26.333061 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application [None req-32562bdf-c9b4-409f-8bb8-8220b54725e5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 14:46:26.333061 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:46:26.333061 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:46:26.333061 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:46:26.333061 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:46:26.333061 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:46:26.333061 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:46:26.333061 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:46:26.333061 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:46:26.333061 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:46:26.333061 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:46:26.333061 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:46:26.333061 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 14:46:26.333061 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 14:46:26.333061 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 14:46:26.333061 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 14:46:26.334030 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:46:26.334030 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:46:26.334030 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:46:26.334030 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:46:26.334030 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:46:26.334030 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:46:26.334030 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:46:26.334030 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application with Lock( Sep 01 14:46:26.334030 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:46:26.334030 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:46:26.334030 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:46:26.334030 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:46:26.334030 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:46:26.334030 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:46:26.334030 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:46:26.334030 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:46:26.334030 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 14:46:26.334977 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 14:46:26.334977 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 14:46:26.334977 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Sep 01 14:46:26.334977 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 352/728] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 1 14:46:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:46:26.340190 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b900f9-b1db-4bf0-af4a-cb7d7ea08dfb None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:26.340190 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b900f9-b1db-4bf0-af4a-cb7d7ea08dfb None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:26.340190 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b900f9-b1db-4bf0-af4a-cb7d7ea08dfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:26.357174 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-d6b900f9-b1db-4bf0-af4a-cb7d7ea08dfb None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:46:26.358460 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-d6b900f9-b1db-4bf0-af4a-cb7d7ea08dfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:46:26.389465 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:26.414824 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:26.428050 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e301b16b-7982-49f0-9f15-a7f615b6d373 None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:26.440126 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 353/729] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 1 14:46:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 14:46:26.440126 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b900f9-b1db-4bf0-af4a-cb7d7ea08dfb None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:26.496081 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba787978-07f7-460a-aa85-9cfd1ad65d44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:26.496691 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba787978-07f7-460a-aa85-9cfd1ad65d44 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:26.496972 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba787978-07f7-460a-aa85-9cfd1ad65d44 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:26.497240 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba787978-07f7-460a-aa85-9cfd1ad65d44 None admin] PATH_INFO: `/v3/roles` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:26.497910 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba787978-07f7-460a-aa85-9cfd1ad65d44 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:46:26.498210 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba787978-07f7-460a-aa85-9cfd1ad65d44 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:26.499609 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba787978-07f7-460a-aa85-9cfd1ad65d44 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:26.513372 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 377/730] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Sep 1 14:46:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 14:46:26.519107 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba787978-07f7-460a-aa85-9cfd1ad65d44 None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:26.604805 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b900f9-b1db-4bf0-af4a-cb7d7ea08dfb None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:26.668429 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0587872a-72c9-4cc4-9ec8-eb85d86606af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:26.669041 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0587872a-72c9-4cc4-9ec8-eb85d86606af None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:26.669317 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0587872a-72c9-4cc4-9ec8-eb85d86606af None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:26.670481 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0587872a-72c9-4cc4-9ec8-eb85d86606af None admin] PATH_INFO: `/v3/domains` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:26.671136 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0587872a-72c9-4cc4-9ec8-eb85d86606af None admin] RBAC: Adding query filter params (name=Default) {{(pid=81341) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:46:26.671436 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0587872a-72c9-4cc4-9ec8-eb85d86606af None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:26.672821 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0587872a-72c9-4cc4-9ec8-eb85d86606af None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:26.690449 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 354/731] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Sep 1 14:46:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:46:26.691790 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89199922-d854-4e7f-84a0-3b94404093e1 None None] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:26.692082 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89199922-d854-4e7f-84a0-3b94404093e1 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:26.692356 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89199922-d854-4e7f-84a0-3b94404093e1 None None] PATH_INFO: `/` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:26.693803 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 355/732] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 1 14:46:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:46:26.696372 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89199922-d854-4e7f-84a0-3b94404093e1 None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:26.702701 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba787978-07f7-460a-aa85-9cfd1ad65d44 None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:26.739607 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f24f4e6-5267-4ad4-8407-79fe8b35c9c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:26.740240 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f24f4e6-5267-4ad4-8407-79fe8b35c9c1 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:26.740518 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f24f4e6-5267-4ad4-8407-79fe8b35c9c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:26.740787 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f24f4e6-5267-4ad4-8407-79fe8b35c9c1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:26.744112 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:26.753010 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:26.756892 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f24f4e6-5267-4ad4-8407-79fe8b35c9c1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:26.758833 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f24f4e6-5267-4ad4-8407-79fe8b35c9c1 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:26.774710 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application [None req-8f24f4e6-5267-4ad4-8407-79fe8b35c9c1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 14:46:26.774710 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:46:26.774710 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:46:26.774710 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:46:26.774710 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:46:26.774710 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:46:26.774710 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:46:26.774710 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:46:26.774710 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:46:26.774710 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:46:26.774710 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:46:26.774710 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:46:26.774710 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 14:46:26.774710 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 14:46:26.774710 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 14:46:26.774710 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 14:46:26.775743 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:46:26.775743 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:46:26.775743 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:46:26.775743 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:46:26.775743 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:46:26.775743 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:46:26.775743 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:46:26.775743 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application with Lock( Sep 01 14:46:26.775743 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:46:26.775743 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:46:26.775743 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:46:26.775743 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:46:26.775743 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:46:26.775743 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:46:26.775743 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:46:26.775743 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:46:26.775743 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 14:46:26.776665 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 14:46:26.776665 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 14:46:26.776665 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Sep 01 14:46:26.776665 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 378/733] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 1 14:46:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:46:26.777753 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6551482-81b2-4f1a-8d06-472ff13a2828 None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:26.777753 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6551482-81b2-4f1a-8d06-472ff13a2828 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:26.781981 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6551482-81b2-4f1a-8d06-472ff13a2828 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:26.796793 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-a6551482-81b2-4f1a-8d06-472ff13a2828 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:46:26.797985 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-a6551482-81b2-4f1a-8d06-472ff13a2828 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:46:26.813205 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89199922-d854-4e7f-84a0-3b94404093e1 None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:26.846225 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addd482b-303b-4e7d-91bf-3305e37a769e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:26.846869 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addd482b-303b-4e7d-91bf-3305e37a769e None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:26.847144 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addd482b-303b-4e7d-91bf-3305e37a769e None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:26.847412 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addd482b-303b-4e7d-91bf-3305e37a769e None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:26.855149 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addd482b-303b-4e7d-91bf-3305e37a769e None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:26.856535 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addd482b-303b-4e7d-91bf-3305e37a769e None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:26.862536 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 379/734] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 1 14:46:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 14:46:26.864494 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdd5e3d-ec84-45ab-ac7f-b4aa3a12355c None None] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:26.864779 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdd5e3d-ec84-45ab-ac7f-b4aa3a12355c None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:26.865046 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdd5e3d-ec84-45ab-ac7f-b4aa3a12355c None None] PATH_INFO: `/` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:26.866427 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 380/735] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 1 14:46:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:46:26.866943 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdd5e3d-ec84-45ab-ac7f-b4aa3a12355c None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:26.868299 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application [None req-addd482b-303b-4e7d-91bf-3305e37a769e None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 01 14:46:26.868299 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:46:26.868299 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:46:26.868299 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:46:26.868299 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:46:26.868299 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:46:26.868299 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:46:26.868299 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:46:26.868299 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:46:26.868299 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:46:26.868299 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:46:26.868299 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:46:26.868299 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 14:46:26.868299 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 14:46:26.868299 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 14:46:26.868299 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 14:46:26.869203 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:46:26.869203 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:46:26.869203 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 14:46:26.869203 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 14:46:26.869203 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 14:46:26.869203 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 14:46:26.869203 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:46:26.869203 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:46:26.869203 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:46:26.869203 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:46:26.869203 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:46:26.869203 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application with Lock( Sep 01 14:46:26.869203 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:46:26.869203 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:46:26.869203 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:46:26.869203 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:46:26.869203 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:46:26.869203 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:46:26.874956 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:46:26.874956 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:46:26.874956 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 14:46:26.874956 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 14:46:26.874956 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 14:46:26.874956 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 14:46:26.874956 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 14:46:26.874956 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 14:46:26.874956 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 01 14:46:26.874956 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Sep 01 14:46:26.874956 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 356/736] 10.222.0.33 () {56 vars in 1117 bytes} [Thu Sep 1 14:46:26 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 14:46:26.874956 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addd482b-303b-4e7d-91bf-3305e37a769e None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:26.973699 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addd482b-303b-4e7d-91bf-3305e37a769e None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:26.974061 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdd5e3d-ec84-45ab-ac7f-b4aa3a12355c None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:26.980549 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:26.991647 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:27.008522 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3408ee-09ee-47e5-a561-4bce57ddf9c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:27.009125 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3408ee-09ee-47e5-a561-4bce57ddf9c6 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:27.009398 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3408ee-09ee-47e5-a561-4bce57ddf9c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:27.009662 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3408ee-09ee-47e5-a561-4bce57ddf9c6 None admin] PATH_INFO: `/v3/domains` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:27.010321 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3408ee-09ee-47e5-a561-4bce57ddf9c6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:46:27.010514 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96568a6-937b-4c8b-b0bf-121af7e7870f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:27.010628 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3408ee-09ee-47e5-a561-4bce57ddf9c6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:27.011137 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96568a6-937b-4c8b-b0bf-121af7e7870f None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:27.011416 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96568a6-937b-4c8b-b0bf-121af7e7870f None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:27.011684 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96568a6-937b-4c8b-b0bf-121af7e7870f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:27.011978 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3408ee-09ee-47e5-a561-4bce57ddf9c6 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:27.025204 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96568a6-937b-4c8b-b0bf-121af7e7870f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 14:46:27.030610 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application [None req-b96568a6-937b-4c8b-b0bf-121af7e7870f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 14:46:27.030610 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:46:27.030610 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:46:27.030610 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:46:27.030610 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:46:27.030610 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:46:27.030610 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:46:27.030610 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:46:27.030610 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:46:27.030610 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:46:27.030610 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:46:27.030610 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:46:27.030610 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 14:46:27.030610 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 14:46:27.030610 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 14:46:27.030610 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 14:46:27.031544 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 14:46:27.031544 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 14:46:27.031544 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 14:46:27.031544 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 14:46:27.031544 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:46:27.031544 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:46:27.031544 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:46:27.031544 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:46:27.031544 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:46:27.031544 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:46:27.031544 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:46:27.031544 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application with Lock( Sep 01 14:46:27.031544 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:46:27.031544 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:46:27.031544 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:46:27.031544 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:46:27.032989 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:46:27.032989 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:46:27.032989 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:46:27.032989 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:46:27.032989 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 14:46:27.032989 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 14:46:27.032989 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 14:46:27.032989 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 14:46:27.032989 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 14:46:27.032989 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 14:46:27.032989 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 14:46:27.032989 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Sep 01 14:46:27.032989 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 357/738] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Sep 1 14:46:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:46:27.033723 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf85c38-c9f5-4d1e-95df-c832ebb9a242 None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:27.033723 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf85c38-c9f5-4d1e-95df-c832ebb9a242 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:27.033723 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf85c38-c9f5-4d1e-95df-c832ebb9a242 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:27.034198 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 381/738] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Sep 1 14:46:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:46:27.035286 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3408ee-09ee-47e5-a561-4bce57ddf9c6 None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:27.051730 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-bcf85c38-c9f5-4d1e-95df-c832ebb9a242 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:46:27.052929 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-bcf85c38-c9f5-4d1e-95df-c832ebb9a242 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:46:27.134345 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 358/739] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 1 14:46:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 14:46:27.134345 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf85c38-c9f5-4d1e-95df-c832ebb9a242 None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:27.146340 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3408ee-09ee-47e5-a561-4bce57ddf9c6 None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:27.182561 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ebc0a0-4c11-412a-aaa4-a464e04b4f3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:27.183166 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ebc0a0-4c11-412a-aaa4-a464e04b4f3c None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:27.183448 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ebc0a0-4c11-412a-aaa4-a464e04b4f3c None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:27.183729 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ebc0a0-4c11-412a-aaa4-a464e04b4f3c None admin] PATH_INFO: `/v3/users` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:27.184442 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ebc0a0-4c11-412a-aaa4-a464e04b4f3c None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:46:27.184747 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ebc0a0-4c11-412a-aaa4-a464e04b4f3c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:27.186164 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ebc0a0-4c11-412a-aaa4-a464e04b4f3c None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:27.203813 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 382/740] 10.222.0.33 () {56 vars in 1149 bytes} [Thu Sep 1 14:46:27 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:46:27.204607 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ebc0a0-4c11-412a-aaa4-a464e04b4f3c None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:27.236757 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:27.243620 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf85c38-c9f5-4d1e-95df-c832ebb9a242 None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:27.247188 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:27.283055 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb24a9ff-11e0-458d-a24a-73a84bb3572d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:27.284260 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb24a9ff-11e0-458d-a24a-73a84bb3572d None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:27.284534 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb24a9ff-11e0-458d-a24a-73a84bb3572d None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:27.284803 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb24a9ff-11e0-458d-a24a-73a84bb3572d None admin] PATH_INFO: `/v3/users/swift` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:27.293120 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb24a9ff-11e0-458d-a24a-73a84bb3572d None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:27.294596 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb24a9ff-11e0-458d-a24a-73a84bb3572d None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:27.306097 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application [None req-eb24a9ff-11e0-458d-a24a-73a84bb3572d None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 01 14:46:27.306097 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:46:27.306097 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:46:27.306097 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:46:27.306097 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:46:27.306097 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:46:27.306097 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:46:27.306097 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:46:27.306097 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:46:27.306097 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:46:27.306097 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:46:27.306097 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:46:27.306097 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 14:46:27.306097 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 14:46:27.306097 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 14:46:27.306097 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 14:46:27.307122 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:46:27.307122 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:46:27.307122 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 14:46:27.307122 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 14:46:27.307122 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 14:46:27.307122 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 14:46:27.307122 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:46:27.307122 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:46:27.307122 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:46:27.307122 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:46:27.307122 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:46:27.307122 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application with Lock( Sep 01 14:46:27.307122 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:46:27.307122 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:46:27.307122 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:46:27.307122 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:46:27.307122 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:46:27.307122 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:46:27.308266 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:46:27.308266 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:46:27.308266 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 14:46:27.308266 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 14:46:27.308266 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 14:46:27.308266 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 14:46:27.308266 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 14:46:27.308266 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 14:46:27.308266 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 01 14:46:27.308266 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Sep 01 14:46:27.312081 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 359/741] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Sep 1 14:46:27 2022] GET /identity/v3/users/swift => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:46:27.312724 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb24a9ff-11e0-458d-a24a-73a84bb3572d None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:27.322378 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ebc0a0-4c11-412a-aaa4-a464e04b4f3c None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:27.358722 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3692d6c2-936c-4156-b2a0-6e205a5e44b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:27.359747 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3692d6c2-936c-4156-b2a0-6e205a5e44b0 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:27.360290 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3692d6c2-936c-4156-b2a0-6e205a5e44b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:27.360760 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3692d6c2-936c-4156-b2a0-6e205a5e44b0 None admin] PATH_INFO: `/v3/roles` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:27.361618 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3692d6c2-936c-4156-b2a0-6e205a5e44b0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:46:27.362132 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3692d6c2-936c-4156-b2a0-6e205a5e44b0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:27.363979 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3692d6c2-936c-4156-b2a0-6e205a5e44b0 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:27.377977 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 383/742] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Sep 1 14:46:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:46:27.379165 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3692d6c2-936c-4156-b2a0-6e205a5e44b0 None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:27.412243 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb24a9ff-11e0-458d-a24a-73a84bb3572d None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:27.445633 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765e96be-3500-4ac3-8e45-ce447e0a358d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:27.446251 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765e96be-3500-4ac3-8e45-ce447e0a358d None admin] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:27.446536 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765e96be-3500-4ac3-8e45-ce447e0a358d None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:27.446809 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765e96be-3500-4ac3-8e45-ce447e0a358d None admin] PATH_INFO: `/v3/services` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:27.447450 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765e96be-3500-4ac3-8e45-ce447e0a358d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:27.448843 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765e96be-3500-4ac3-8e45-ce447e0a358d None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:27.465061 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 360/743] 10.222.0.33 () {60 vars in 1141 bytes} [Thu Sep 1 14:46:27 2022] POST /identity/v3/services => generated 240 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:46:27.465061 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765e96be-3500-4ac3-8e45-ce447e0a358d None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:27.479583 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:27.483721 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3692d6c2-936c-4156-b2a0-6e205a5e44b0 None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:27.486698 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:27.541503 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4a8448-2429-456f-806b-8fe51c160864 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:27.542209 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4a8448-2429-456f-806b-8fe51c160864 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:27.542520 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4a8448-2429-456f-806b-8fe51c160864 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:27.542789 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4a8448-2429-456f-806b-8fe51c160864 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:27.587082 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765e96be-3500-4ac3-8e45-ce447e0a358d None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:27.622800 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4a8448-2429-456f-806b-8fe51c160864 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:27.624432 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4a8448-2429-456f-806b-8fe51c160864 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:27.625031 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffd3ed4-4d8a-421e-b0b8-2a25e0f016e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:27.625633 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffd3ed4-4d8a-421e-b0b8-2a25e0f016e4 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:27.625911 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffd3ed4-4d8a-421e-b0b8-2a25e0f016e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:27.626328 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffd3ed4-4d8a-421e-b0b8-2a25e0f016e4 None admin] PATH_INFO: `/v3/users` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:27.627591 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffd3ed4-4d8a-421e-b0b8-2a25e0f016e4 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81341) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:46:27.627591 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffd3ed4-4d8a-421e-b0b8-2a25e0f016e4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:27.630450 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffd3ed4-4d8a-421e-b0b8-2a25e0f016e4 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:27.647721 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 361/744] 10.222.0.33 () {56 vars in 1135 bytes} [Thu Sep 1 14:46:27 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:46:27.648569 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffd3ed4-4d8a-421e-b0b8-2a25e0f016e4 None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:27.727170 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:27.729569 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:27.735288 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 384/745] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Sep 1 14:46:27 2022] GET /identity/v3/auth/tokens => generated 898 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 14:46:27.736429 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4a8448-2429-456f-806b-8fe51c160864 None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:27.759203 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffd3ed4-4d8a-421e-b0b8-2a25e0f016e4 None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:27.794485 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fefb0d-0b29-4078-be5b-5a77d91558c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:27.795143 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fefb0d-0b29-4078-be5b-5a77d91558c4 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:27.795426 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fefb0d-0b29-4078-be5b-5a77d91558c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:27.795690 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fefb0d-0b29-4078-be5b-5a77d91558c4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:27.807775 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fefb0d-0b29-4078-be5b-5a77d91558c4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:27.809411 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fefb0d-0b29-4078-be5b-5a77d91558c4 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:27.825234 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application [None req-36fefb0d-0b29-4078-be5b-5a77d91558c4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 14:46:27.825234 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:46:27.825234 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:46:27.825234 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:46:27.825234 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:46:27.825234 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:46:27.825234 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:46:27.825234 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:46:27.825234 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:46:27.825234 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:46:27.825234 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:46:27.825234 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:46:27.825234 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 14:46:27.825234 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 14:46:27.825234 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 14:46:27.825234 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 14:46:27.826621 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:46:27.826621 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:46:27.826621 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:46:27.826621 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:46:27.826621 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:46:27.826621 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:46:27.826621 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:46:27.826621 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application with Lock( Sep 01 14:46:27.826621 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:46:27.826621 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:46:27.826621 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:46:27.826621 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:46:27.826621 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:46:27.826621 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:46:27.826621 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:46:27.826621 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:46:27.826621 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 14:46:27.827602 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 14:46:27.827602 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 14:46:27.827602 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Sep 01 14:46:27.827602 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 362/746] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 1 14:46:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:46:27.827602 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fefb0d-0b29-4078-be5b-5a77d91558c4 None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:27.849205 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4a8448-2429-456f-806b-8fe51c160864 None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:27.886057 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411a9a94-de35-4c30-b72a-122142a03cad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:27.886989 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411a9a94-de35-4c30-b72a-122142a03cad None admin] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:27.887456 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411a9a94-de35-4c30-b72a-122142a03cad None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:27.887897 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411a9a94-de35-4c30-b72a-122142a03cad None admin] PATH_INFO: `/v3/services` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:27.888711 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411a9a94-de35-4c30-b72a-122142a03cad None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:27.890360 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411a9a94-de35-4c30-b72a-122142a03cad None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:27.906138 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 385/747] 10.222.0.33 () {60 vars in 1141 bytes} [Thu Sep 1 14:46:27 2022] POST /identity/v3/services => generated 251 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:46:27.907210 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411a9a94-de35-4c30-b72a-122142a03cad None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:27.954523 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fefb0d-0b29-4078-be5b-5a77d91558c4 None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:27.981846 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:27.999060 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:28.026792 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fda406-28e8-4b72-9cc1-0cea7ed4b8ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:28.028070 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fda406-28e8-4b72-9cc1-0cea7ed4b8ce None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:28.028587 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fda406-28e8-4b72-9cc1-0cea7ed4b8ce None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:28.029275 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fda406-28e8-4b72-9cc1-0cea7ed4b8ce None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:28.040997 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411a9a94-de35-4c30-b72a-122142a03cad None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:28.083786 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f41d9e2-4c3e-4400-ad27-1e0292ec5033 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:28.084722 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f41d9e2-4c3e-4400-ad27-1e0292ec5033 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:28.085261 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f41d9e2-4c3e-4400-ad27-1e0292ec5033 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:28.085732 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f41d9e2-4c3e-4400-ad27-1e0292ec5033 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:28.102651 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f41d9e2-4c3e-4400-ad27-1e0292ec5033 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:28.104132 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f41d9e2-4c3e-4400-ad27-1e0292ec5033 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:28.114139 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fda406-28e8-4b72-9cc1-0cea7ed4b8ce None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:28.116119 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fda406-28e8-4b72-9cc1-0cea7ed4b8ce None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:28.123974 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application [None req-0f41d9e2-4c3e-4400-ad27-1e0292ec5033 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 14:46:28.123974 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:46:28.123974 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:46:28.123974 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:46:28.123974 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:46:28.123974 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:46:28.123974 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:46:28.123974 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:46:28.123974 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:46:28.123974 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:46:28.123974 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:46:28.123974 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:46:28.123974 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 14:46:28.123974 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 14:46:28.123974 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 14:46:28.123974 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 14:46:28.125016 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:46:28.125016 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:46:28.125016 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:46:28.125016 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:46:28.125016 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:46:28.125016 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:46:28.125016 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:46:28.125016 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application with Lock( Sep 01 14:46:28.125016 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:46:28.125016 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:46:28.125016 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:46:28.125016 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:46:28.125016 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:46:28.125016 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:46:28.125016 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:46:28.125016 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:46:28.125016 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 14:46:28.126147 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 14:46:28.126147 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 14:46:28.126147 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Sep 01 14:46:28.126602 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 386/748] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 1 14:46:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:46:28.127589 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f41d9e2-4c3e-4400-ad27-1e0292ec5033 None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:28.228682 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 363/749] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Sep 1 14:46:27 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 14:46:28.229792 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fda406-28e8-4b72-9cc1-0cea7ed4b8ce None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:28.236684 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f41d9e2-4c3e-4400-ad27-1e0292ec5033 None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:28.245560 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:28.257968 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:28.280472 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffecfa04-77ac-44dc-9925-37eb7f0b6c1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:28.281236 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffecfa04-77ac-44dc-9925-37eb7f0b6c1d None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:28.281611 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffecfa04-77ac-44dc-9925-37eb7f0b6c1d None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:28.282342 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffecfa04-77ac-44dc-9925-37eb7f0b6c1d None admin] PATH_INFO: `/v3/domains` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:28.283096 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffecfa04-77ac-44dc-9925-37eb7f0b6c1d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:46:28.283521 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffecfa04-77ac-44dc-9925-37eb7f0b6c1d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:28.285646 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffecfa04-77ac-44dc-9925-37eb7f0b6c1d None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:28.302314 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 387/750] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Sep 1 14:46:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:46:28.303369 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffecfa04-77ac-44dc-9925-37eb7f0b6c1d None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:28.359126 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fda406-28e8-4b72-9cc1-0cea7ed4b8ce None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:28.398911 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933f69ee-1756-48b8-aedb-7da0f9cbfa28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:28.399700 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933f69ee-1756-48b8-aedb-7da0f9cbfa28 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:28.400075 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933f69ee-1756-48b8-aedb-7da0f9cbfa28 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:28.400441 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933f69ee-1756-48b8-aedb-7da0f9cbfa28 None admin] PATH_INFO: `/v3/domains` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:28.401222 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933f69ee-1756-48b8-aedb-7da0f9cbfa28 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81341) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:46:28.401627 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933f69ee-1756-48b8-aedb-7da0f9cbfa28 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:28.403187 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933f69ee-1756-48b8-aedb-7da0f9cbfa28 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:28.419020 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 364/751] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Sep 1 14:46:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:46:28.420036 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933f69ee-1756-48b8-aedb-7da0f9cbfa28 None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:28.514062 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:28.531081 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffecfa04-77ac-44dc-9925-37eb7f0b6c1d None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:28.535306 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933f69ee-1756-48b8-aedb-7da0f9cbfa28 None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:28.567735 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8c1a74-2f08-4ba8-bf18-4dbdae650886 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:28.568652 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8c1a74-2f08-4ba8-bf18-4dbdae650886 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:28.569113 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8c1a74-2f08-4ba8-bf18-4dbdae650886 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:28.569589 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8c1a74-2f08-4ba8-bf18-4dbdae650886 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:28.573523 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fd3ce4-16bb-4fa0-87a0-d963f2ade0ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:28.574264 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fd3ce4-16bb-4fa0-87a0-d963f2ade0ac None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:28.574613 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fd3ce4-16bb-4fa0-87a0-d963f2ade0ac None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:28.574953 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fd3ce4-16bb-4fa0-87a0-d963f2ade0ac None admin] PATH_INFO: `/v3/users/nova` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:28.582957 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8c1a74-2f08-4ba8-bf18-4dbdae650886 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:28.584372 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16fd3ce4-16bb-4fa0-87a0-d963f2ade0ac None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:28.584993 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8c1a74-2f08-4ba8-bf18-4dbdae650886 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:28.586205 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16fd3ce4-16bb-4fa0-87a0-d963f2ade0ac None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:28.599075 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application [None req-16fd3ce4-16bb-4fa0-87a0-d963f2ade0ac None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 01 14:46:28.599075 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:46:28.599075 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:46:28.599075 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:46:28.599075 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:46:28.599075 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:46:28.599075 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:46:28.599075 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:46:28.599075 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:46:28.599075 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:46:28.599075 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:46:28.599075 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:46:28.599075 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 14:46:28.599075 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 14:46:28.599075 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 14:46:28.599075 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 14:46:28.600106 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:46:28.600106 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:46:28.600106 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 14:46:28.600106 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 14:46:28.600106 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 14:46:28.600106 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 14:46:28.600106 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:46:28.600106 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:46:28.600106 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:46:28.600106 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:46:28.600106 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:46:28.600106 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application with Lock( Sep 01 14:46:28.600106 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:46:28.600106 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:46:28.600106 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:46:28.600106 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:46:28.600106 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:46:28.600106 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:46:28.601085 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:46:28.601085 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:46:28.601085 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 14:46:28.601085 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 14:46:28.601085 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 14:46:28.601085 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 14:46:28.601085 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 14:46:28.601085 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 14:46:28.601085 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 01 14:46:28.601085 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Sep 01 14:46:28.604868 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 365/752] 10.222.0.33 () {56 vars in 1093 bytes} [Thu Sep 1 14:46:28 2022] GET /identity/v3/users/nova => generated 82 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:46:28.605687 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fd3ce4-16bb-4fa0-87a0-d963f2ade0ac None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:28.607266 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application [None req-ce8c1a74-2f08-4ba8-bf18-4dbdae650886 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 14:46:28.607266 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:46:28.607266 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:46:28.607266 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:46:28.607266 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:46:28.607266 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:46:28.607266 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:46:28.607266 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:46:28.607266 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:46:28.607266 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:46:28.607266 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:46:28.607266 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:46:28.607266 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 14:46:28.607266 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 14:46:28.607266 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 14:46:28.607266 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 14:46:28.608221 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:46:28.608221 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:46:28.608221 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:46:28.608221 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:46:28.608221 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:46:28.608221 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:46:28.608221 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:46:28.608221 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application with Lock( Sep 01 14:46:28.608221 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:46:28.608221 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:46:28.608221 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:46:28.608221 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:46:28.608221 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:46:28.608221 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:46:28.608221 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:46:28.608221 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:46:28.608221 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 14:46:28.609141 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 14:46:28.609141 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 14:46:28.609141 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Sep 01 14:46:28.611444 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 388/753] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 1 14:46:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 308 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:46:28.612437 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8c1a74-2f08-4ba8-bf18-4dbdae650886 None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:28.633057 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:28.715553 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fd3ce4-16bb-4fa0-87a0-d963f2ade0ac None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:28.717154 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8c1a74-2f08-4ba8-bf18-4dbdae650886 None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:28.751580 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b8e27a-5f7d-44b5-83ab-0f5d3f092ade None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:28.752199 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b8e27a-5f7d-44b5-83ab-0f5d3f092ade None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:28.752474 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b8e27a-5f7d-44b5-83ab-0f5d3f092ade None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:28.752741 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b8e27a-5f7d-44b5-83ab-0f5d3f092ade None admin] PATH_INFO: `/v3/projects/service` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:28.753003 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eab3dd7-f9ef-4711-945c-3e4627f5905f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:28.753954 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eab3dd7-f9ef-4711-945c-3e4627f5905f None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:28.755094 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eab3dd7-f9ef-4711-945c-3e4627f5905f None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:28.755570 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eab3dd7-f9ef-4711-945c-3e4627f5905f None admin] PATH_INFO: `/v3/users` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:28.756524 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eab3dd7-f9ef-4711-945c-3e4627f5905f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:46:28.757059 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eab3dd7-f9ef-4711-945c-3e4627f5905f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:28.758780 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eab3dd7-f9ef-4711-945c-3e4627f5905f None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:28.760110 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b8e27a-5f7d-44b5-83ab-0f5d3f092ade None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:28.762269 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b8e27a-5f7d-44b5-83ab-0f5d3f092ade None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:28.776748 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application [None req-94b8e27a-5f7d-44b5-83ab-0f5d3f092ade None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 14:46:28.776748 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:46:28.776748 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:46:28.776748 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:46:28.776748 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:46:28.776748 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:46:28.776748 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:46:28.776748 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:46:28.776748 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:46:28.776748 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:46:28.776748 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:46:28.776748 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:46:28.776748 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 14:46:28.776748 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 14:46:28.776748 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 14:46:28.777730 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 14:46:28.777730 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:46:28.777730 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:46:28.777730 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:46:28.777730 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:46:28.777730 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:46:28.777730 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:46:28.777730 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:46:28.777730 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application with Lock( Sep 01 14:46:28.777730 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:46:28.777730 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:46:28.777730 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:46:28.777730 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:46:28.777730 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:46:28.777730 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:46:28.777730 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:46:28.777730 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:46:28.778739 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 14:46:28.778739 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 14:46:28.778739 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 14:46:28.778739 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 14:46:28.778739 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 14:46:28.778739 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 14:46:28.778739 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 14:46:28.778739 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Sep 01 14:46:28.778739 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 366/754] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Sep 1 14:46:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:46:28.779239 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b8e27a-5f7d-44b5-83ab-0f5d3f092ade None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:28.782360 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:28.785779 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 389/755] 10.222.0.33 () {56 vars in 1133 bytes} [Thu Sep 1 14:46:28 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:46:28.786865 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eab3dd7-f9ef-4711-945c-3e4627f5905f None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:28.871342 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:28.892439 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b8e27a-5f7d-44b5-83ab-0f5d3f092ade None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:28.897030 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eab3dd7-f9ef-4711-945c-3e4627f5905f None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:28.929351 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccdae8b-a02d-44a4-9b1f-07fa9910db50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:28.930257 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccdae8b-a02d-44a4-9b1f-07fa9910db50 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:28.930719 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccdae8b-a02d-44a4-9b1f-07fa9910db50 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:28.931162 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccdae8b-a02d-44a4-9b1f-07fa9910db50 None admin] PATH_INFO: `/v3/domains` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:28.932094 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccdae8b-a02d-44a4-9b1f-07fa9910db50 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81341) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:46:28.932625 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccdae8b-a02d-44a4-9b1f-07fa9910db50 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:28.934356 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccdae8b-a02d-44a4-9b1f-07fa9910db50 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:28.934942 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bd79da-3321-413c-9813-8010a95d6013 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:28.935618 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bd79da-3321-413c-9813-8010a95d6013 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:28.935979 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bd79da-3321-413c-9813-8010a95d6013 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:28.936324 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bd79da-3321-413c-9813-8010a95d6013 None admin] PATH_INFO: `/v3/projects` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:28.937181 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95bd79da-3321-413c-9813-8010a95d6013 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:46:28.937554 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95bd79da-3321-413c-9813-8010a95d6013 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:28.939179 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95bd79da-3321-413c-9813-8010a95d6013 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:28.949006 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 367/756] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Sep 1 14:46:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:46:28.950205 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccdae8b-a02d-44a4-9b1f-07fa9910db50 None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:28.952766 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 390/757] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Sep 1 14:46:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:46:28.954603 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bd79da-3321-413c-9813-8010a95d6013 None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:29.015164 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:29.058438 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccdae8b-a02d-44a4-9b1f-07fa9910db50 None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:29.069761 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bd79da-3321-413c-9813-8010a95d6013 None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:29.096303 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8929277-9174-4b5e-a636-dc18c3e66854 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:29.096941 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8929277-9174-4b5e-a636-dc18c3e66854 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:29.097299 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8929277-9174-4b5e-a636-dc18c3e66854 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:29.097499 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8929277-9174-4b5e-a636-dc18c3e66854 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:29.109254 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8929277-9174-4b5e-a636-dc18c3e66854 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:29.110924 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8929277-9174-4b5e-a636-dc18c3e66854 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:29.112690 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d2686e-629e-4172-afd1-6421a8c93b40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:29.114439 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d2686e-629e-4172-afd1-6421a8c93b40 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:29.114985 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d2686e-629e-4172-afd1-6421a8c93b40 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:29.115462 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d2686e-629e-4172-afd1-6421a8c93b40 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:29.118516 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:29.123491 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d2686e-629e-4172-afd1-6421a8c93b40 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:29.125114 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d2686e-629e-4172-afd1-6421a8c93b40 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:29.129589 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application [None req-d8929277-9174-4b5e-a636-dc18c3e66854 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 14:46:29.129589 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:46:29.129589 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:46:29.129589 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:46:29.129589 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:46:29.129589 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:46:29.129589 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:46:29.129589 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:46:29.129589 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:46:29.129589 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:46:29.129589 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:46:29.129589 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:46:29.129589 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 14:46:29.129589 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 14:46:29.129589 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 14:46:29.129589 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 14:46:29.130631 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:46:29.130631 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:46:29.130631 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:46:29.130631 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:46:29.130631 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:46:29.130631 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:46:29.130631 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:46:29.130631 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application with Lock( Sep 01 14:46:29.130631 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:46:29.130631 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:46:29.130631 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:46:29.130631 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:46:29.130631 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:46:29.130631 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:46:29.130631 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:46:29.130631 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:46:29.130631 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 14:46:29.131550 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 14:46:29.131550 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 14:46:29.131550 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Sep 01 14:46:29.137882 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 368/758] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 1 14:46:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:46:29.138248 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application [None req-58d2686e-629e-4172-afd1-6421a8c93b40 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 14:46:29.138248 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:46:29.138248 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:46:29.138248 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:46:29.138248 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:46:29.138248 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:46:29.138248 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:46:29.138248 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:46:29.138248 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:46:29.138248 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:46:29.138248 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:46:29.138248 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:46:29.138248 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 14:46:29.138248 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 14:46:29.138248 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 14:46:29.139132 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 14:46:29.139132 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:46:29.139132 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:46:29.139132 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:46:29.139132 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:46:29.139132 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:46:29.139132 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:46:29.139132 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:46:29.139132 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application with Lock( Sep 01 14:46:29.139132 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:46:29.139132 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:46:29.139132 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:46:29.139132 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:46:29.139132 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:46:29.139132 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:46:29.139132 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:46:29.139132 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:46:29.140069 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 14:46:29.140069 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 14:46:29.140069 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 14:46:29.140069 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 14:46:29.140069 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 14:46:29.140069 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 14:46:29.140069 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 14:46:29.140069 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Sep 01 14:46:29.140069 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 391/759] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Sep 1 14:46:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:46:29.140586 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d2686e-629e-4172-afd1-6421a8c93b40 None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:29.142003 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8929277-9174-4b5e-a636-dc18c3e66854 None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:29.247117 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8929277-9174-4b5e-a636-dc18c3e66854 None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:29.249223 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d2686e-629e-4172-afd1-6421a8c93b40 None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:29.262095 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:29.287125 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7d5b07-d18a-4403-9517-6f453c027a44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:29.287935 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7d5b07-d18a-4403-9517-6f453c027a44 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:29.288359 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7d5b07-d18a-4403-9517-6f453c027a44 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:29.288708 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7d5b07-d18a-4403-9517-6f453c027a44 None admin] PATH_INFO: `/v3/domains` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:29.289521 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7d5b07-d18a-4403-9517-6f453c027a44 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81341) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:46:29.289810 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbe2f6f-5501-4efa-aebe-8e4bf6e41c9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:29.289941 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7d5b07-d18a-4403-9517-6f453c027a44 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:29.290725 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbe2f6f-5501-4efa-aebe-8e4bf6e41c9f None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:29.291187 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbe2f6f-5501-4efa-aebe-8e4bf6e41c9f None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:29.291531 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7d5b07-d18a-4403-9517-6f453c027a44 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:29.291818 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbe2f6f-5501-4efa-aebe-8e4bf6e41c9f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:29.292726 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbe2f6f-5501-4efa-aebe-8e4bf6e41c9f None admin] RBAC: Adding query filter params (role.id=5e3f992e804046f381b3de8bf2a512e2, scope.project.id=e8c85b4bd17e4e89b56bf0f831b474d5, user.id=24b6f275ffb04ffeae3c2097bfd040f5) {{(pid=81342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:46:29.293404 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbe2f6f-5501-4efa-aebe-8e4bf6e41c9f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:29.295504 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbe2f6f-5501-4efa-aebe-8e4bf6e41c9f None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:29.302341 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 392/760] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Sep 1 14:46:29 2022] GET /identity/v3/role_assignments?user.id=24b6f275ffb04ffeae3c2097bfd040f5&scope.project.id=e8c85b4bd17e4e89b56bf0f831b474d5&role.id=5e3f992e804046f381b3de8bf2a512e2 => generated 259 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:46:29.303387 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbe2f6f-5501-4efa-aebe-8e4bf6e41c9f None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:29.310338 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 369/761] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Sep 1 14:46:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:46:29.311906 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7d5b07-d18a-4403-9517-6f453c027a44 None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:29.382336 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:29.456795 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7d5b07-d18a-4403-9517-6f453c027a44 None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:29.480112 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbe2f6f-5501-4efa-aebe-8e4bf6e41c9f None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:29.507964 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40beda57-4461-494e-9f4a-1cc0fdac641c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:29.507964 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40beda57-4461-494e-9f4a-1cc0fdac641c None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:29.507964 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40beda57-4461-494e-9f4a-1cc0fdac641c None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:29.507964 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40beda57-4461-494e-9f4a-1cc0fdac641c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:29.515643 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40beda57-4461-494e-9f4a-1cc0fdac641c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:29.517110 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40beda57-4461-494e-9f4a-1cc0fdac641c None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:29.529106 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc166478-1cda-44fe-a489-c19f9cf8d373 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:29.529290 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application [None req-40beda57-4461-494e-9f4a-1cc0fdac641c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 14:46:29.529290 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:46:29.529290 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:46:29.529290 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:46:29.529290 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:46:29.529290 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:46:29.529290 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:46:29.529290 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:46:29.529290 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:46:29.529290 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:46:29.529290 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:46:29.529290 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:46:29.529290 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 14:46:29.529290 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 14:46:29.529290 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 14:46:29.530171 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 14:46:29.530171 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:46:29.530171 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:46:29.530171 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:46:29.530171 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:46:29.530171 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:46:29.530171 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:46:29.530171 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:46:29.530171 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application with Lock( Sep 01 14:46:29.530171 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:46:29.530171 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:46:29.530171 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:46:29.530171 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:46:29.530171 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:46:29.530171 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:46:29.530171 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:46:29.530171 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:46:29.531154 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 14:46:29.531154 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 14:46:29.531154 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 14:46:29.531154 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 14:46:29.531154 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 14:46:29.531154 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 14:46:29.531154 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 14:46:29.531154 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Sep 01 14:46:29.531598 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc166478-1cda-44fe-a489-c19f9cf8d373 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:29.531598 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc166478-1cda-44fe-a489-c19f9cf8d373 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:29.531598 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc166478-1cda-44fe-a489-c19f9cf8d373 None admin] PATH_INFO: `/v3/projects` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:29.531598 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc166478-1cda-44fe-a489-c19f9cf8d373 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:46:29.531598 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc166478-1cda-44fe-a489-c19f9cf8d373 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:29.534007 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc166478-1cda-44fe-a489-c19f9cf8d373 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:29.535102 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 370/762] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Sep 1 14:46:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 14:46:29.536927 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cd0ab7-c9bc-4eef-8286-c4fb4353847c None None] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:29.537210 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cd0ab7-c9bc-4eef-8286-c4fb4353847c None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:29.537481 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cd0ab7-c9bc-4eef-8286-c4fb4353847c None None] PATH_INFO: `/` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:29.539184 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 371/763] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 1 14:46:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:46:29.539876 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cd0ab7-c9bc-4eef-8286-c4fb4353847c None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:29.554338 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 393/764] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Sep 1 14:46:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:46:29.555642 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3b4c74-a2b4-4580-a90f-6fb5bda0e438 None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:29.555924 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3b4c74-a2b4-4580-a90f-6fb5bda0e438 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:29.556180 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3b4c74-a2b4-4580-a90f-6fb5bda0e438 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:29.575417 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-4d3b4c74-a2b4-4580-a90f-6fb5bda0e438 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:46:29.576734 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-4d3b4c74-a2b4-4580-a90f-6fb5bda0e438 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:46:29.584518 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:29.650066 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 394/765] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 1 14:46:29 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 14:46:29.656376 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3b4c74-a2b4-4580-a90f-6fb5bda0e438 None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:29.666954 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cd0ab7-c9bc-4eef-8286-c4fb4353847c None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:29.717070 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:29.727908 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f279a260-f9bb-499e-a20f-9e0f4e2b8f34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:29.728524 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f279a260-f9bb-499e-a20f-9e0f4e2b8f34 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:29.728805 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f279a260-f9bb-499e-a20f-9e0f4e2b8f34 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:29.729079 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f279a260-f9bb-499e-a20f-9e0f4e2b8f34 None admin] PATH_INFO: `/v3/projects` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:29.729789 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f279a260-f9bb-499e-a20f-9e0f4e2b8f34 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81341) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:46:29.730093 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f279a260-f9bb-499e-a20f-9e0f4e2b8f34 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:29.731648 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f279a260-f9bb-499e-a20f-9e0f4e2b8f34 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:29.755685 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 372/766] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Sep 1 14:46:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:46:29.760381 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f279a260-f9bb-499e-a20f-9e0f4e2b8f34 None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:29.773678 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3b4c74-a2b4-4580-a90f-6fb5bda0e438 None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:29.811308 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36914d2-19bd-48f9-ac8f-6195224543d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:29.812228 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36914d2-19bd-48f9-ac8f-6195224543d3 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:29.812605 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36914d2-19bd-48f9-ac8f-6195224543d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:29.812997 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36914d2-19bd-48f9-ac8f-6195224543d3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:29.813825 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36914d2-19bd-48f9-ac8f-6195224543d3 None admin] RBAC: Adding query filter params (role.id=2a2adb769e55476b82e5014897e0a9c0, scope.project.id=e8c85b4bd17e4e89b56bf0f831b474d5, user.id=73a5bda5a6ea46e9863ccafd57dee339) {{(pid=81342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:46:29.814316 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36914d2-19bd-48f9-ac8f-6195224543d3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:29.815953 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36914d2-19bd-48f9-ac8f-6195224543d3 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:29.822801 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 395/767] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Sep 1 14:46:29 2022] GET /identity/v3/role_assignments?user.id=73a5bda5a6ea46e9863ccafd57dee339&scope.project.id=e8c85b4bd17e4e89b56bf0f831b474d5&role.id=2a2adb769e55476b82e5014897e0a9c0 => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:46:29.893224 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:29.898856 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f279a260-f9bb-499e-a20f-9e0f4e2b8f34 None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:29.934557 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda9570f-9927-45c8-99f1-cf68ec66c1b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:29.935242 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda9570f-9927-45c8-99f1-cf68ec66c1b5 None admin] REQUEST_METHOD: `PUT` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:29.935520 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda9570f-9927-45c8-99f1-cf68ec66c1b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:29.935790 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda9570f-9927-45c8-99f1-cf68ec66c1b5 None admin] PATH_INFO: `/v3/projects/e8c85b4bd17e4e89b56bf0f831b474d5/users/4858bb8621854ec8aba836e7d9050498/roles/2a2adb769e55476b82e5014897e0a9c0` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:29.937856 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda9570f-9927-45c8-99f1-cf68ec66c1b5 None admin] RBAC: Authorizing `identity:create_grant(project_id=e8c85b4bd17e4e89b56bf0f831b474d5, user_id=4858bb8621854ec8aba836e7d9050498, role_id=2a2adb769e55476b82e5014897e0a9c0)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:29.939467 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda9570f-9927-45c8-99f1-cf68ec66c1b5 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:29.966176 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 373/768] 10.222.0.33 () {58 vars in 1439 bytes} [Thu Sep 1 14:46:29 2022] PUT /identity/v3/projects/e8c85b4bd17e4e89b56bf0f831b474d5/users/4858bb8621854ec8aba836e7d9050498/roles/2a2adb769e55476b82e5014897e0a9c0 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:46:29.983021 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8317c1-b038-400f-80dd-a7da5e0bae51 None None] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:29.983363 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8317c1-b038-400f-80dd-a7da5e0bae51 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:29.983576 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8317c1-b038-400f-80dd-a7da5e0bae51 None None] PATH_INFO: `/` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:29.985253 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 396/769] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 1 14:46:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:46:29.991478 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdccc24-b153-4325-8021-39df40db37e1 None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:29.991794 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdccc24-b153-4325-8021-39df40db37e1 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:29.992027 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdccc24-b153-4325-8021-39df40db37e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:30.016500 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-2fdccc24-b153-4325-8021-39df40db37e1 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:46:30.017724 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-2fdccc24-b153-4325-8021-39df40db37e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:46:30.115283 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 374/770] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 1 14:46:29 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 14:46:30.135690 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8317c1-b038-400f-80dd-a7da5e0bae51 None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:30.257227 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8317c1-b038-400f-80dd-a7da5e0bae51 None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:30.271309 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:30.298459 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b14d29b-ab77-4122-97c0-15c712184a33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:30.299141 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b14d29b-ab77-4122-97c0-15c712184a33 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:30.299416 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b14d29b-ab77-4122-97c0-15c712184a33 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:30.299687 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b14d29b-ab77-4122-97c0-15c712184a33 None admin] PATH_INFO: `/v3/services/network` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:30.300281 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b14d29b-ab77-4122-97c0-15c712184a33 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:30.302229 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b14d29b-ab77-4122-97c0-15c712184a33 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:30.315239 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application [None req-3b14d29b-ab77-4122-97c0-15c712184a33 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 01 14:46:30.315239 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:46:30.315239 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 14:46:30.315239 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 14:46:30.315239 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 14:46:30.315239 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 14:46:30.315239 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 14:46:30.315239 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 14:46:30.315239 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 01 14:46:30.315239 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Sep 01 14:46:30.315239 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 14:46:30.315239 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Sep 01 14:46:30.315239 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:46:30.315239 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:46:30.315239 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:46:30.315239 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:46:30.315239 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:46:30.316238 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:46:30.316238 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:46:30.316238 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:46:30.316238 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:46:30.316238 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:46:30.316238 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:46:30.316238 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 14:46:30.316238 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 14:46:30.316238 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 14:46:30.316238 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 14:46:30.316238 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:46:30.316238 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:46:30.316238 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:46:30.316238 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:46:30.316238 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:46:30.316238 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:46:30.318038 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:46:30.318038 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application with Lock( Sep 01 14:46:30.318038 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:46:30.318038 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:46:30.318038 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:46:30.318038 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:46:30.318038 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:46:30.318038 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:46:30.318038 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:46:30.318038 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:46:30.318038 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 14:46:30.318038 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 14:46:30.318038 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 01 14:46:30.318038 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Sep 01 14:46:30.318038 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 397/771] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Sep 1 14:46:30 2022] GET /identity/v3/services/network => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:46:30.325263 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdccc24-b153-4325-8021-39df40db37e1 None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:30.433171 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:30.435809 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b14d29b-ab77-4122-97c0-15c712184a33 None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:30.466734 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdccc24-b153-4325-8021-39df40db37e1 None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:30.537262 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbeb1cd-8215-4f1f-a047-7cf60640f8e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:30.538870 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbeb1cd-8215-4f1f-a047-7cf60640f8e5 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:30.539527 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbeb1cd-8215-4f1f-a047-7cf60640f8e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:30.541192 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbeb1cd-8215-4f1f-a047-7cf60640f8e5 None admin] PATH_INFO: `/v3/services` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:30.542620 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbeb1cd-8215-4f1f-a047-7cf60640f8e5 None admin] RBAC: Adding query filter params (name=network) {{(pid=81341) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:46:30.543363 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbeb1cd-8215-4f1f-a047-7cf60640f8e5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:30.545587 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbeb1cd-8215-4f1f-a047-7cf60640f8e5 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:30.556502 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 375/772] 10.222.0.33 () {56 vars in 1112 bytes} [Thu Sep 1 14:46:30 2022] GET /identity/v3/services?name=network => generated 124 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:46:30.569113 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbeb1cd-8215-4f1f-a047-7cf60640f8e5 None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:30.620304 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b14d29b-ab77-4122-97c0-15c712184a33 None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:30.670939 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cdd654-829c-425f-bdef-9d40b83145c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:30.671578 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cdd654-829c-425f-bdef-9d40b83145c7 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:30.671855 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cdd654-829c-425f-bdef-9d40b83145c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:30.672126 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cdd654-829c-425f-bdef-9d40b83145c7 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:30.672713 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34cdd654-829c-425f-bdef-9d40b83145c7 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:30.674274 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34cdd654-829c-425f-bdef-9d40b83145c7 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:30.692038 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application [None req-34cdd654-829c-425f-bdef-9d40b83145c7 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 01 14:46:30.692038 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:46:30.692038 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 14:46:30.692038 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 14:46:30.692038 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 14:46:30.692038 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 14:46:30.692038 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 14:46:30.692038 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 14:46:30.692038 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 01 14:46:30.692038 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Sep 01 14:46:30.692038 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 14:46:30.692038 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Sep 01 14:46:30.692038 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:46:30.692038 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:46:30.692038 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:46:30.692038 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:46:30.692038 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:46:30.693071 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:46:30.693071 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:46:30.693071 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:46:30.693071 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:46:30.693071 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:46:30.693071 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:46:30.693071 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 14:46:30.693071 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 14:46:30.693071 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 14:46:30.693071 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 14:46:30.693071 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:46:30.693071 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:46:30.693071 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:46:30.693071 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:46:30.693071 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:46:30.693071 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:46:30.693952 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:46:30.693952 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application with Lock( Sep 01 14:46:30.693952 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:46:30.693952 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:46:30.693952 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:46:30.693952 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:46:30.693952 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:46:30.693952 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:46:30.693952 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:46:30.693952 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:46:30.693952 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 14:46:30.693952 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 14:46:30.693952 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 01 14:46:30.693952 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Sep 01 14:46:30.699834 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 398/773] 10.222.0.33 () {56 vars in 1129 bytes} [Thu Sep 1 14:46:30 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:46:30.704879 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cdd654-829c-425f-bdef-9d40b83145c7 None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:30.726259 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbeb1cd-8215-4f1f-a047-7cf60640f8e5 None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:30.768847 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:30.778068 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c46f858-0d96-49d2-849d-ab67e24209e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:30.779027 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c46f858-0d96-49d2-849d-ab67e24209e4 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:30.779198 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c46f858-0d96-49d2-849d-ab67e24209e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:30.779575 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c46f858-0d96-49d2-849d-ab67e24209e4 None admin] PATH_INFO: `/v3/services` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:30.780310 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c46f858-0d96-49d2-849d-ab67e24209e4 None admin] RBAC: Adding query filter params (type=network) {{(pid=81341) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:46:30.780709 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c46f858-0d96-49d2-849d-ab67e24209e4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:30.782200 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c46f858-0d96-49d2-849d-ab67e24209e4 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:30.798380 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 376/774] 10.222.0.33 () {56 vars in 1112 bytes} [Thu Sep 1 14:46:30 2022] GET /identity/v3/services?type=network => generated 350 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:46:30.819311 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c46f858-0d96-49d2-849d-ab67e24209e4 None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:30.840426 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cdd654-829c-425f-bdef-9d40b83145c7 None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:30.845061 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:30.890934 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dda78b7-ed86-4325-a656-7fcacca002ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:30.891929 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dda78b7-ed86-4325-a656-7fcacca002ee None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:30.892444 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dda78b7-ed86-4325-a656-7fcacca002ee None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:30.892917 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dda78b7-ed86-4325-a656-7fcacca002ee None admin] PATH_INFO: `/v3/services` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:30.893803 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dda78b7-ed86-4325-a656-7fcacca002ee None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:46:30.894327 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dda78b7-ed86-4325-a656-7fcacca002ee None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:30.896022 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dda78b7-ed86-4325-a656-7fcacca002ee None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:30.901411 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 399/775] 10.222.0.33 () {56 vars in 1124 bytes} [Thu Sep 1 14:46:30 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 14:46:30.906520 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dda78b7-ed86-4325-a656-7fcacca002ee None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:31.012387 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c46f858-0d96-49d2-849d-ab67e24209e4 None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:31.044430 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dda78b7-ed86-4325-a656-7fcacca002ee None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:31.053176 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:31.080914 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5f445e-7175-4e2e-8d0e-1f570319eb2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:31.086091 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5f445e-7175-4e2e-8d0e-1f570319eb2d None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:31.086091 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5f445e-7175-4e2e-8d0e-1f570319eb2d None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:31.086091 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5f445e-7175-4e2e-8d0e-1f570319eb2d None admin] PATH_INFO: `/v3/endpoints` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:31.086091 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5f445e-7175-4e2e-8d0e-1f570319eb2d None admin] RBAC: Adding query filter params (interface=public, service_id=fa4186b3f3df413a86381859c1dd67e3, region_id=RegionOne) {{(pid=81341) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:46:31.086091 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5f445e-7175-4e2e-8d0e-1f570319eb2d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:31.086091 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5f445e-7175-4e2e-8d0e-1f570319eb2d None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:31.091777 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2417a835-dfbf-4bd1-8e99-9bf9193c460e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:31.092574 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 377/776] 10.222.0.33 () {56 vars in 1251 bytes} [Thu Sep 1 14:46:30 2022] GET /identity/v3/endpoints?service_id=fa4186b3f3df413a86381859c1dd67e3&interface=public®ion_id=RegionOne => generated 194 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:46:31.092960 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2417a835-dfbf-4bd1-8e99-9bf9193c460e None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:31.093496 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2417a835-dfbf-4bd1-8e99-9bf9193c460e None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:31.094054 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2417a835-dfbf-4bd1-8e99-9bf9193c460e None admin] PATH_INFO: `/v3/services` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:31.094904 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2417a835-dfbf-4bd1-8e99-9bf9193c460e None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:46:31.095436 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2417a835-dfbf-4bd1-8e99-9bf9193c460e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:31.097409 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2417a835-dfbf-4bd1-8e99-9bf9193c460e None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:31.099450 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5f445e-7175-4e2e-8d0e-1f570319eb2d None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:31.103816 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 400/777] 10.222.0.33 () {56 vars in 1124 bytes} [Thu Sep 1 14:46:30 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 14:46:31.109779 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2417a835-dfbf-4bd1-8e99-9bf9193c460e None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:31.235538 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:31.279442 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5f445e-7175-4e2e-8d0e-1f570319eb2d None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:31.305816 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2417a835-dfbf-4bd1-8e99-9bf9193c460e None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:31.350521 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bbf4f7-c2c2-40b7-aab6-41aaa88d1fb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:31.351138 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bbf4f7-c2c2-40b7-aab6-41aaa88d1fb8 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:31.351420 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bbf4f7-c2c2-40b7-aab6-41aaa88d1fb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:31.351693 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bbf4f7-c2c2-40b7-aab6-41aaa88d1fb8 None admin] PATH_INFO: `/v3/services` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:31.352304 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36bbf4f7-c2c2-40b7-aab6-41aaa88d1fb8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:31.353700 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36bbf4f7-c2c2-40b7-aab6-41aaa88d1fb8 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:31.369546 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 378/778] 10.222.0.33 () {56 vars in 1087 bytes} [Thu Sep 1 14:46:31 2022] GET /identity/v3/services => generated 772 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:46:31.372924 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccf59d1-57e8-425e-8d63-d9e1ccbc5210 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:31.373551 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccf59d1-57e8-425e-8d63-d9e1ccbc5210 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:31.373833 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccf59d1-57e8-425e-8d63-d9e1ccbc5210 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:31.374106 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccf59d1-57e8-425e-8d63-d9e1ccbc5210 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:31.374935 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccf59d1-57e8-425e-8d63-d9e1ccbc5210 None admin] RBAC: Adding query filter params (interface=public, service_id=f515bebeac054553a0fb4e214922cab7, region_id=RegionOne) {{(pid=81342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:46:31.375252 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccf59d1-57e8-425e-8d63-d9e1ccbc5210 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:31.376679 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccf59d1-57e8-425e-8d63-d9e1ccbc5210 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:31.385962 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 401/779] 10.222.0.33 () {56 vars in 1251 bytes} [Thu Sep 1 14:46:31 2022] GET /identity/v3/endpoints?service_id=f515bebeac054553a0fb4e214922cab7&interface=public®ion_id=RegionOne => generated 194 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 14:46:31.391917 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bbf4f7-c2c2-40b7-aab6-41aaa88d1fb8 None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:31.531686 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bbf4f7-c2c2-40b7-aab6-41aaa88d1fb8 None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:31.547047 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb316dc-2310-496d-953d-668d43e6853c None None] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:31.547355 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb316dc-2310-496d-953d-668d43e6853c None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:31.547599 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb316dc-2310-496d-953d-668d43e6853c None None] PATH_INFO: `/` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:31.548941 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 402/780] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 1 14:46:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 14:46:31.555482 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff88702-047e-404d-86ce-d496e8dd877e None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:31.555755 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff88702-047e-404d-86ce-d496e8dd877e None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:31.556084 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff88702-047e-404d-86ce-d496e8dd877e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:31.592466 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-cff88702-047e-404d-86ce-d496e8dd877e None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:46:31.593706 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-cff88702-047e-404d-86ce-d496e8dd877e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:46:31.601849 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f716a7-a39c-4c13-9924-94e78774909d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:31.602504 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f716a7-a39c-4c13-9924-94e78774909d None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:31.602788 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f716a7-a39c-4c13-9924-94e78774909d None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:31.603058 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f716a7-a39c-4c13-9924-94e78774909d None admin] PATH_INFO: `/v3/services` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:31.603671 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f716a7-a39c-4c13-9924-94e78774909d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:31.605085 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f716a7-a39c-4c13-9924-94e78774909d None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:31.622990 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 379/781] 10.222.0.33 () {56 vars in 1087 bytes} [Thu Sep 1 14:46:31 2022] GET /identity/v3/services => generated 772 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:46:31.641530 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:31.690430 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 403/782] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 1 14:46:31 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 14:46:31.973625 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f716a7-a39c-4c13-9924-94e78774909d None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:31.977811 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:32.090358 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f716a7-a39c-4c13-9924-94e78774909d None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:32.128390 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8cc139-a87d-41ef-a19f-3d4e2ddd75ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:32.129032 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8cc139-a87d-41ef-a19f-3d4e2ddd75ec None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:32.129857 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8cc139-a87d-41ef-a19f-3d4e2ddd75ec None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:32.130131 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8cc139-a87d-41ef-a19f-3d4e2ddd75ec None admin] PATH_INFO: `/v3/roles/service` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:32.150398 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8cc139-a87d-41ef-a19f-3d4e2ddd75ec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 14:46:32.155982 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application [None req-3f8cc139-a87d-41ef-a19f-3d4e2ddd75ec None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 14:46:32.155982 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:46:32.155982 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:46:32.155982 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:46:32.155982 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:46:32.155982 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:46:32.155982 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:46:32.155982 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:46:32.155982 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:46:32.155982 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:46:32.155982 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:46:32.155982 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:46:32.155982 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 14:46:32.155982 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 14:46:32.155982 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 14:46:32.155982 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 14:46:32.156953 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 14:46:32.156953 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 14:46:32.156953 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 14:46:32.156953 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 14:46:32.156953 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:46:32.156953 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:46:32.156953 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:46:32.156953 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:46:32.156953 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:46:32.156953 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:46:32.156953 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:46:32.156953 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application with Lock( Sep 01 14:46:32.156953 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:46:32.156953 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:46:32.156953 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:46:32.156953 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:46:32.158512 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:46:32.158512 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:46:32.158512 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:46:32.158512 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:46:32.158512 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 14:46:32.158512 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 14:46:32.158512 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 14:46:32.158512 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 14:46:32.158512 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 14:46:32.158512 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 14:46:32.158512 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 14:46:32.158512 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Sep 01 14:46:32.158512 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 380/783] 10.222.0.33 () {56 vars in 1102 bytes} [Thu Sep 1 14:46:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:46:32.162939 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff88702-047e-404d-86ce-d496e8dd877e None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:32.246295 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd05ca7c-0d19-4883-b9e5-bd77f52be9a7 None None] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:32.246592 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd05ca7c-0d19-4883-b9e5-bd77f52be9a7 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:32.246775 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd05ca7c-0d19-4883-b9e5-bd77f52be9a7 None None] PATH_INFO: `/` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:32.248413 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 381/784] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 1 14:46:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:46:32.260023 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f57aa63-584c-4c62-b610-9f92860465e1 None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:32.260308 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f57aa63-584c-4c62-b610-9f92860465e1 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:32.260582 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f57aa63-584c-4c62-b610-9f92860465e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:32.287299 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-0f57aa63-584c-4c62-b610-9f92860465e1 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:46:32.288860 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-0f57aa63-584c-4c62-b610-9f92860465e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:46:32.293378 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff88702-047e-404d-86ce-d496e8dd877e None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:32.332363 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af57730d-793c-485f-b1c6-455c69f2ab56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:32.333007 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af57730d-793c-485f-b1c6-455c69f2ab56 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:32.333237 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af57730d-793c-485f-b1c6-455c69f2ab56 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:32.333511 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af57730d-793c-485f-b1c6-455c69f2ab56 None admin] PATH_INFO: `/v3/roles` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:32.334177 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af57730d-793c-485f-b1c6-455c69f2ab56 None admin] RBAC: Adding query filter params (name=service) {{(pid=81342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:46:32.334522 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af57730d-793c-485f-b1c6-455c69f2ab56 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:32.335932 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af57730d-793c-485f-b1c6-455c69f2ab56 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:32.352466 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 404/785] 10.222.0.33 () {56 vars in 1103 bytes} [Thu Sep 1 14:46:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:46:32.353744 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af57730d-793c-485f-b1c6-455c69f2ab56 None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:32.368454 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:32.377218 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 382/786] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 1 14:46:32 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 14:46:32.383364 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8646ff3b-aad1-4473-b3f9-c4b22dbfcbc7 None None] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:32.386630 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8646ff3b-aad1-4473-b3f9-c4b22dbfcbc7 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:32.386985 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8646ff3b-aad1-4473-b3f9-c4b22dbfcbc7 None None] PATH_INFO: `/` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:32.387121 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:32.388754 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 383/787] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 1 14:46:32 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:46:32.394849 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fbd802-4ed2-4432-83cb-963cb851f361 None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:32.395200 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fbd802-4ed2-4432-83cb-963cb851f361 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:32.395600 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fbd802-4ed2-4432-83cb-963cb851f361 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:32.428078 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-73fbd802-4ed2-4432-83cb-963cb851f361 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:46:32.429360 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-73fbd802-4ed2-4432-83cb-963cb851f361 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:46:32.468963 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af57730d-793c-485f-b1c6-455c69f2ab56 None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:32.522995 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 384/788] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 1 14:46:32 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 14:46:32.523188 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a03a6fb-f3f3-4552-8955-2752714f22b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:32.523587 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a03a6fb-f3f3-4552-8955-2752714f22b0 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:32.523869 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a03a6fb-f3f3-4552-8955-2752714f22b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:32.524140 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a03a6fb-f3f3-4552-8955-2752714f22b0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:32.547243 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a03a6fb-f3f3-4552-8955-2752714f22b0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:32.548713 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a03a6fb-f3f3-4552-8955-2752714f22b0 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:32.578878 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application [None req-9a03a6fb-f3f3-4552-8955-2752714f22b0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 14:46:32.578878 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:46:32.578878 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:46:32.578878 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:46:32.578878 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:46:32.578878 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:46:32.578878 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:46:32.578878 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:46:32.578878 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:46:32.578878 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:46:32.578878 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:46:32.578878 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:46:32.578878 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 14:46:32.578878 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 14:46:32.578878 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 14:46:32.578878 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 14:46:32.579940 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:46:32.579940 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:46:32.579940 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:46:32.579940 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:46:32.579940 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:46:32.579940 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:46:32.579940 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:46:32.579940 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application with Lock( Sep 01 14:46:32.579940 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:46:32.579940 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:46:32.579940 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:46:32.579940 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:46:32.579940 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:46:32.579940 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:46:32.579940 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:46:32.579940 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:46:32.579940 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 14:46:32.581345 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 14:46:32.581345 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 14:46:32.581345 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Sep 01 14:46:32.581345 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 405/789] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 1 14:46:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:46:32.585744 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fbd802-4ed2-4432-83cb-963cb851f361 None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:32.759609 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fbd802-4ed2-4432-83cb-963cb851f361 None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:32.779997 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a03a6fb-f3f3-4552-8955-2752714f22b0 None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:32.806606 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:32.816187 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf574448-41ca-40d8-884a-226fdbe543af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:32.817267 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf574448-41ca-40d8-884a-226fdbe543af None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:32.817772 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf574448-41ca-40d8-884a-226fdbe543af None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:32.818262 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf574448-41ca-40d8-884a-226fdbe543af None admin] PATH_INFO: `/v3/domains` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:32.819140 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf574448-41ca-40d8-884a-226fdbe543af None admin] RBAC: Adding query filter params (name=Default) {{(pid=81341) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:46:32.819652 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf574448-41ca-40d8-884a-226fdbe543af None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:32.821688 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf574448-41ca-40d8-884a-226fdbe543af None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:32.839027 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 385/790] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Sep 1 14:46:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:46:32.839876 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf574448-41ca-40d8-884a-226fdbe543af None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:32.890624 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:32.898917 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a03a6fb-f3f3-4552-8955-2752714f22b0 None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:32.951274 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170fd453-84ba-4e84-b598-e2577376e1fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:32.952078 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170fd453-84ba-4e84-b598-e2577376e1fc None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:32.952453 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170fd453-84ba-4e84-b598-e2577376e1fc None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:32.952820 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170fd453-84ba-4e84-b598-e2577376e1fc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:32.968288 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170fd453-84ba-4e84-b598-e2577376e1fc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 14:46:32.968785 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf574448-41ca-40d8-884a-226fdbe543af None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:32.974879 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application [None req-170fd453-84ba-4e84-b598-e2577376e1fc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 14:46:32.974879 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:46:32.974879 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:46:32.974879 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:46:32.974879 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:46:32.974879 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:46:32.974879 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:46:32.974879 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:46:32.974879 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:46:32.974879 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:46:32.974879 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:46:32.974879 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:46:32.974879 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 14:46:32.974879 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 14:46:32.974879 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 14:46:32.974879 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 14:46:32.975996 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 14:46:32.975996 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 14:46:32.975996 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 14:46:32.975996 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 14:46:32.975996 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:46:32.975996 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:46:32.975996 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:46:32.975996 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:46:32.975996 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:46:32.975996 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:46:32.975996 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:46:32.975996 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application with Lock( Sep 01 14:46:32.975996 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:46:32.975996 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:46:32.975996 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:46:32.975996 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:46:32.977116 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:46:32.977116 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:46:32.977116 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:46:32.977116 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:46:32.977116 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 14:46:32.977116 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 14:46:32.977116 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 14:46:32.977116 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 14:46:32.977116 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 14:46:32.977116 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 14:46:32.977116 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 14:46:32.977116 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Sep 01 14:46:32.981081 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 406/791] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Sep 1 14:46:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:46:32.981823 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170fd453-84ba-4e84-b598-e2577376e1fc None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:33.010792 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8213b7d-93c8-4506-9c33-a8471b28958b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:33.011575 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8213b7d-93c8-4506-9c33-a8471b28958b None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:33.011969 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8213b7d-93c8-4506-9c33-a8471b28958b None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:33.012238 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8213b7d-93c8-4506-9c33-a8471b28958b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:33.031533 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8213b7d-93c8-4506-9c33-a8471b28958b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 14:46:33.038800 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application [None req-e8213b7d-93c8-4506-9c33-a8471b28958b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 14:46:33.038800 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:46:33.038800 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:46:33.038800 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:46:33.038800 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:46:33.038800 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:46:33.038800 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:46:33.038800 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:46:33.038800 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:46:33.038800 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:46:33.038800 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:46:33.038800 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:46:33.038800 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 14:46:33.038800 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 14:46:33.038800 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 14:46:33.038800 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 14:46:33.040167 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 14:46:33.040167 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 14:46:33.040167 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 14:46:33.040167 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 14:46:33.040167 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:46:33.040167 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:46:33.040167 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:46:33.040167 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:46:33.040167 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:46:33.040167 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:46:33.040167 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:46:33.040167 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application with Lock( Sep 01 14:46:33.040167 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:46:33.040167 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:46:33.040167 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:46:33.040167 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:46:33.041565 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:46:33.041565 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:46:33.041565 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:46:33.041565 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:46:33.041565 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 14:46:33.041565 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 14:46:33.041565 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 14:46:33.041565 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 14:46:33.041565 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 14:46:33.041565 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 14:46:33.041565 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 14:46:33.041565 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Sep 01 14:46:33.041565 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 386/792] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Sep 1 14:46:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:46:33.041565 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8213b7d-93c8-4506-9c33-a8471b28958b None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:33.101046 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:33.102968 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170fd453-84ba-4e84-b598-e2577376e1fc None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:33.151383 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889ce33d-5935-4151-aecb-4100bf6c14ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:33.152735 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889ce33d-5935-4151-aecb-4100bf6c14ca None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:33.153433 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889ce33d-5935-4151-aecb-4100bf6c14ca None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:33.153876 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889ce33d-5935-4151-aecb-4100bf6c14ca None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:33.158170 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8213b7d-93c8-4506-9c33-a8471b28958b None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:33.164188 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889ce33d-5935-4151-aecb-4100bf6c14ca None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:33.166301 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889ce33d-5935-4151-aecb-4100bf6c14ca None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:33.168621 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:33.183923 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application [None req-889ce33d-5935-4151-aecb-4100bf6c14ca None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 01 14:46:33.183923 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:46:33.183923 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:46:33.183923 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:46:33.183923 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:46:33.183923 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:46:33.183923 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:46:33.183923 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:46:33.183923 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:46:33.183923 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:46:33.183923 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:46:33.183923 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:46:33.183923 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 14:46:33.183923 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 14:46:33.183923 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 14:46:33.183923 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 14:46:33.184894 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:46:33.184894 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:46:33.184894 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 14:46:33.184894 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 14:46:33.184894 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 14:46:33.184894 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 14:46:33.184894 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:46:33.184894 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:46:33.184894 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:46:33.184894 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:46:33.184894 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:46:33.184894 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application with Lock( Sep 01 14:46:33.184894 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:46:33.184894 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:46:33.184894 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:46:33.184894 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:46:33.184894 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:46:33.184894 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:46:33.185876 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:46:33.185876 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:46:33.185876 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 14:46:33.185876 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 14:46:33.185876 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 14:46:33.185876 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 14:46:33.185876 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 14:46:33.185876 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 14:46:33.185876 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 01 14:46:33.185876 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Sep 01 14:46:33.190551 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 407/793] 10.222.0.33 () {56 vars in 1117 bytes} [Thu Sep 1 14:46:32 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:46:33.191409 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889ce33d-5935-4151-aecb-4100bf6c14ca None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:33.203086 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f8f5da-2513-41c6-81b7-9a888b2f6acf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:33.203975 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f8f5da-2513-41c6-81b7-9a888b2f6acf None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:33.204769 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f8f5da-2513-41c6-81b7-9a888b2f6acf None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:33.205219 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f8f5da-2513-41c6-81b7-9a888b2f6acf None admin] PATH_INFO: `/v3/roles` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:33.206454 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f8f5da-2513-41c6-81b7-9a888b2f6acf None admin] RBAC: Adding query filter params (name=admin) {{(pid=81341) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:46:33.207707 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f8f5da-2513-41c6-81b7-9a888b2f6acf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:33.209499 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f8f5da-2513-41c6-81b7-9a888b2f6acf None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:33.220708 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 387/794] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Sep 1 14:46:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:46:33.221745 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f8f5da-2513-41c6-81b7-9a888b2f6acf None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:33.308676 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889ce33d-5935-4151-aecb-4100bf6c14ca None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:33.331185 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f8f5da-2513-41c6-81b7-9a888b2f6acf None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:33.339197 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:33.347955 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63186708-b120-42d3-a67f-6b0ab6fe64a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:33.349148 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63186708-b120-42d3-a67f-6b0ab6fe64a0 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:33.349718 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63186708-b120-42d3-a67f-6b0ab6fe64a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:33.350159 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63186708-b120-42d3-a67f-6b0ab6fe64a0 None admin] PATH_INFO: `/v3/roles` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:33.351194 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63186708-b120-42d3-a67f-6b0ab6fe64a0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:46:33.351712 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63186708-b120-42d3-a67f-6b0ab6fe64a0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:33.353277 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63186708-b120-42d3-a67f-6b0ab6fe64a0 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:33.362838 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 408/795] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Sep 1 14:46:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:46:33.364158 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63186708-b120-42d3-a67f-6b0ab6fe64a0 None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:33.369425 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e18223-4150-47a7-9977-aa8eb7ec6d05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:33.370179 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e18223-4150-47a7-9977-aa8eb7ec6d05 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:33.370557 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e18223-4150-47a7-9977-aa8eb7ec6d05 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:33.370903 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e18223-4150-47a7-9977-aa8eb7ec6d05 None admin] PATH_INFO: `/v3/users` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:33.371726 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e18223-4150-47a7-9977-aa8eb7ec6d05 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81341) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:46:33.372098 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e18223-4150-47a7-9977-aa8eb7ec6d05 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:33.373598 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e18223-4150-47a7-9977-aa8eb7ec6d05 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:33.398382 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 388/796] 10.222.0.33 () {56 vars in 1149 bytes} [Thu Sep 1 14:46:33 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:46:33.399728 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e18223-4150-47a7-9977-aa8eb7ec6d05 None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:33.429299 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:33.473765 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63186708-b120-42d3-a67f-6b0ab6fe64a0 None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:33.510161 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0e942e-ff73-463b-92cb-aee086d98f35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:33.511174 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0e942e-ff73-463b-92cb-aee086d98f35 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:33.511732 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0e942e-ff73-463b-92cb-aee086d98f35 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:33.512302 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0e942e-ff73-463b-92cb-aee086d98f35 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:33.515270 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e18223-4150-47a7-9977-aa8eb7ec6d05 None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:33.524415 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0e942e-ff73-463b-92cb-aee086d98f35 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:33.526212 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0e942e-ff73-463b-92cb-aee086d98f35 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:33.541696 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application [None req-9e0e942e-ff73-463b-92cb-aee086d98f35 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 14:46:33.541696 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:46:33.541696 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:46:33.541696 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:46:33.541696 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:46:33.541696 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:46:33.541696 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:46:33.541696 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:46:33.541696 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:46:33.541696 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:46:33.541696 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:46:33.541696 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:46:33.541696 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 14:46:33.541696 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 14:46:33.541696 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 14:46:33.541696 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 14:46:33.542768 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:46:33.542768 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:46:33.542768 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:46:33.542768 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:46:33.542768 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:46:33.542768 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:46:33.542768 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:46:33.542768 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application with Lock( Sep 01 14:46:33.542768 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:46:33.542768 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:46:33.542768 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:46:33.542768 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:46:33.542768 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:46:33.542768 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:46:33.542768 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:46:33.542768 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:46:33.542768 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 14:46:33.543870 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 14:46:33.543870 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 14:46:33.543870 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Sep 01 14:46:33.545649 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 409/797] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 1 14:46:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:46:33.546599 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0e942e-ff73-463b-92cb-aee086d98f35 None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:33.556862 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c692fc8d-a105-436a-83a1-390ffd86e6b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:33.557498 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c692fc8d-a105-436a-83a1-390ffd86e6b6 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:33.557781 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c692fc8d-a105-436a-83a1-390ffd86e6b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:33.558056 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c692fc8d-a105-436a-83a1-390ffd86e6b6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:33.570176 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c692fc8d-a105-436a-83a1-390ffd86e6b6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:33.572098 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c692fc8d-a105-436a-83a1-390ffd86e6b6 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:33.589606 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application [None req-c692fc8d-a105-436a-83a1-390ffd86e6b6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 14:46:33.589606 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:46:33.589606 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:46:33.589606 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:46:33.589606 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:46:33.589606 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:46:33.589606 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:46:33.589606 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:46:33.589606 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:46:33.589606 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:46:33.589606 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:46:33.589606 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:46:33.589606 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 14:46:33.589606 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 14:46:33.589606 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 14:46:33.589606 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 14:46:33.591194 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:46:33.591194 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:46:33.591194 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:46:33.591194 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:46:33.591194 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:46:33.591194 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:46:33.591194 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:46:33.591194 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application with Lock( Sep 01 14:46:33.591194 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:46:33.591194 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:46:33.591194 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:46:33.591194 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:46:33.591194 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:46:33.591194 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:46:33.591194 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:46:33.591194 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:46:33.591194 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 14:46:33.592213 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 14:46:33.592213 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 14:46:33.592213 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Sep 01 14:46:33.592213 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 389/798] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 1 14:46:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:46:33.592213 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c692fc8d-a105-436a-83a1-390ffd86e6b6 None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:33.611358 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:33.657654 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0e942e-ff73-463b-92cb-aee086d98f35 None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:33.672006 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:33.694263 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6b3b55-6d98-42c5-b47d-01a1add22473 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:33.695187 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6b3b55-6d98-42c5-b47d-01a1add22473 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:33.695949 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6b3b55-6d98-42c5-b47d-01a1add22473 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:33.696392 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6b3b55-6d98-42c5-b47d-01a1add22473 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:33.708056 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6b3b55-6d98-42c5-b47d-01a1add22473 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:33.709739 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6b3b55-6d98-42c5-b47d-01a1add22473 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:33.716592 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c692fc8d-a105-436a-83a1-390ffd86e6b6 None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:33.727663 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application [None req-9f6b3b55-6d98-42c5-b47d-01a1add22473 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 14:46:33.727663 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:46:33.727663 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:46:33.727663 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:46:33.727663 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:46:33.727663 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:46:33.727663 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:46:33.727663 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:46:33.727663 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:46:33.727663 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:46:33.727663 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:46:33.727663 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:46:33.727663 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 14:46:33.727663 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 14:46:33.727663 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 14:46:33.727663 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 14:46:33.728628 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:46:33.728628 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:46:33.728628 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:46:33.728628 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:46:33.728628 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:46:33.728628 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:46:33.728628 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:46:33.728628 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application with Lock( Sep 01 14:46:33.728628 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:46:33.728628 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:46:33.728628 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:46:33.728628 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:46:33.728628 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:46:33.728628 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:46:33.728628 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:46:33.728628 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:46:33.728628 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 14:46:33.729544 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 14:46:33.729544 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 14:46:33.729544 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Sep 01 14:46:33.729544 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 410/799] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 1 14:46:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:46:33.729544 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6b3b55-6d98-42c5-b47d-01a1add22473 None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:33.763046 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281904e6-e14c-4615-b25f-d5f2a513d308 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:33.763684 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281904e6-e14c-4615-b25f-d5f2a513d308 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:33.764202 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281904e6-e14c-4615-b25f-d5f2a513d308 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:33.764486 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281904e6-e14c-4615-b25f-d5f2a513d308 None admin] PATH_INFO: `/v3/domains` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:33.765136 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281904e6-e14c-4615-b25f-d5f2a513d308 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81341) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:46:33.765441 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281904e6-e14c-4615-b25f-d5f2a513d308 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:33.767373 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281904e6-e14c-4615-b25f-d5f2a513d308 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:33.783472 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 390/800] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Sep 1 14:46:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:46:33.784702 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281904e6-e14c-4615-b25f-d5f2a513d308 None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:33.849403 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6b3b55-6d98-42c5-b47d-01a1add22473 None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:33.873149 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:33.886220 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ea622d-8ff7-439f-aac5-a506937873f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:33.886876 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ea622d-8ff7-439f-aac5-a506937873f8 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:33.887161 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ea622d-8ff7-439f-aac5-a506937873f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:33.887435 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ea622d-8ff7-439f-aac5-a506937873f8 None admin] PATH_INFO: `/v3/domains` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:33.888097 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ea622d-8ff7-439f-aac5-a506937873f8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:46:33.888395 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ea622d-8ff7-439f-aac5-a506937873f8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:33.889773 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ea622d-8ff7-439f-aac5-a506937873f8 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:33.899339 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281904e6-e14c-4615-b25f-d5f2a513d308 None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:33.902483 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 411/801] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Sep 1 14:46:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:46:33.903304 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ea622d-8ff7-439f-aac5-a506937873f8 None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:33.927706 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:33.934313 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c090d05-af29-4fb0-a0de-5f2950fdd134 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:33.935024 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c090d05-af29-4fb0-a0de-5f2950fdd134 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:33.935402 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c090d05-af29-4fb0-a0de-5f2950fdd134 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:33.935747 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c090d05-af29-4fb0-a0de-5f2950fdd134 None admin] PATH_INFO: `/v3/domains` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:33.936455 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c090d05-af29-4fb0-a0de-5f2950fdd134 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81341) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:46:33.936827 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c090d05-af29-4fb0-a0de-5f2950fdd134 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:33.938281 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c090d05-af29-4fb0-a0de-5f2950fdd134 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:33.950596 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 391/802] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Sep 1 14:46:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:46:33.951742 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546f1e2f-a955-475f-9ee9-596c07185e54 None None] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:33.952065 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546f1e2f-a955-475f-9ee9-596c07185e54 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:33.952338 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546f1e2f-a955-475f-9ee9-596c07185e54 None None] PATH_INFO: `/` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:33.953509 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 392/803] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 1 14:46:33 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:46:33.954526 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15a1d18-99e4-4e33-9196-77a87e95134f None None] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:33.954814 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15a1d18-99e4-4e33-9196-77a87e95134f None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:33.955056 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15a1d18-99e4-4e33-9196-77a87e95134f None None] PATH_INFO: `/` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:33.956464 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 393/804] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 1 14:46:33 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:46:33.957163 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15a1d18-99e4-4e33-9196-77a87e95134f None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:34.006660 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ea622d-8ff7-439f-aac5-a506937873f8 None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:34.040347 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d8310a-e524-4947-a811-a106ea6d040f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:34.040963 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d8310a-e524-4947-a811-a106ea6d040f None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:34.041242 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d8310a-e524-4947-a811-a106ea6d040f None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:34.041514 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d8310a-e524-4947-a811-a106ea6d040f None admin] PATH_INFO: `/v3/users/nova` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:34.048322 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d8310a-e524-4947-a811-a106ea6d040f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:34.049747 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d8310a-e524-4947-a811-a106ea6d040f None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:34.057427 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15a1d18-99e4-4e33-9196-77a87e95134f None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:34.061025 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application [None req-b0d8310a-e524-4947-a811-a106ea6d040f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 01 14:46:34.061025 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:46:34.061025 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:46:34.061025 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:46:34.061025 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:46:34.061025 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:46:34.061025 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:46:34.061025 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:46:34.061025 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:46:34.061025 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:46:34.061025 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:46:34.061025 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:46:34.061025 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 14:46:34.061025 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 14:46:34.061025 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 14:46:34.061025 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 14:46:34.062112 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:46:34.062112 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:46:34.062112 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 14:46:34.062112 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 14:46:34.062112 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 14:46:34.062112 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 14:46:34.062112 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:46:34.062112 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:46:34.062112 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:46:34.062112 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:46:34.062112 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:46:34.062112 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application with Lock( Sep 01 14:46:34.062112 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:46:34.062112 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:46:34.062112 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:46:34.062112 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:46:34.062112 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:46:34.062112 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:46:34.063125 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:46:34.063125 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:46:34.063125 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 14:46:34.063125 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 14:46:34.063125 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 14:46:34.063125 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 14:46:34.063125 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 14:46:34.063125 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 14:46:34.063125 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 01 14:46:34.063125 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Sep 01 14:46:34.063125 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 412/805] 10.222.0.33 () {56 vars in 1093 bytes} [Thu Sep 1 14:46:33 2022] GET /identity/v3/users/nova => generated 82 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:46:34.063125 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d8310a-e524-4947-a811-a106ea6d040f None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:34.093858 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e89029-78d1-44c1-865a-9a3efa58c149 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:34.094634 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e89029-78d1-44c1-865a-9a3efa58c149 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:34.095007 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e89029-78d1-44c1-865a-9a3efa58c149 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:34.095350 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e89029-78d1-44c1-865a-9a3efa58c149 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:34.098003 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:34.103726 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e89029-78d1-44c1-865a-9a3efa58c149 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:34.105325 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e89029-78d1-44c1-865a-9a3efa58c149 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:34.116591 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application [None req-26e89029-78d1-44c1-865a-9a3efa58c149 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 01 14:46:34.116591 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:46:34.116591 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:46:34.116591 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:46:34.116591 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:46:34.116591 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:46:34.116591 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:46:34.116591 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:46:34.116591 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:46:34.116591 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:46:34.116591 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:46:34.116591 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:46:34.116591 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 14:46:34.116591 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 14:46:34.116591 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 14:46:34.116591 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 14:46:34.117715 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:46:34.117715 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:46:34.117715 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 14:46:34.117715 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 14:46:34.117715 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 14:46:34.117715 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 14:46:34.117715 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:46:34.117715 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:46:34.117715 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:46:34.117715 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:46:34.117715 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:46:34.117715 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application with Lock( Sep 01 14:46:34.117715 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:46:34.117715 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:46:34.117715 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:46:34.117715 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:46:34.117715 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:46:34.117715 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:46:34.118915 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:46:34.118915 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:46:34.118915 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 14:46:34.118915 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 14:46:34.118915 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 14:46:34.118915 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 14:46:34.118915 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 14:46:34.118915 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 14:46:34.118915 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 01 14:46:34.118915 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Sep 01 14:46:34.118915 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 394/806] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Sep 1 14:46:33 2022] GET /identity/v3/users/swift => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:46:34.119651 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c0827a-e9b2-4d7a-81ac-22ffb1b9ce3d None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:34.119651 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c0827a-e9b2-4d7a-81ac-22ffb1b9ce3d None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:34.119651 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c0827a-e9b2-4d7a-81ac-22ffb1b9ce3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:34.137203 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-31c0827a-e9b2-4d7a-81ac-22ffb1b9ce3d None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:46:34.138430 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-31c0827a-e9b2-4d7a-81ac-22ffb1b9ce3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:46:34.149667 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:34.165981 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d8310a-e524-4947-a811-a106ea6d040f None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:34.200607 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153ba97a-2523-412e-9ea2-ad89c803e091 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:34.201225 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153ba97a-2523-412e-9ea2-ad89c803e091 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:34.201505 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153ba97a-2523-412e-9ea2-ad89c803e091 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:34.201777 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153ba97a-2523-412e-9ea2-ad89c803e091 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:34.205334 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 395/807] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 1 14:46:34 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 14:46:34.206679 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f23d2e-352d-4d3c-9c85-59403b14eba5 None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:34.206967 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f23d2e-352d-4d3c-9c85-59403b14eba5 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:34.207842 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f23d2e-352d-4d3c-9c85-59403b14eba5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:34.220011 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153ba97a-2523-412e-9ea2-ad89c803e091 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:34.221522 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153ba97a-2523-412e-9ea2-ad89c803e091 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:34.224655 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-30f23d2e-352d-4d3c-9c85-59403b14eba5 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:46:34.225847 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-30f23d2e-352d-4d3c-9c85-59403b14eba5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:46:34.233393 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application [None req-153ba97a-2523-412e-9ea2-ad89c803e091 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 14:46:34.233393 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:46:34.233393 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:46:34.233393 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:46:34.233393 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:46:34.233393 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:46:34.233393 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:46:34.233393 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:46:34.233393 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:46:34.233393 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:46:34.233393 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:46:34.233393 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:46:34.233393 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 14:46:34.233393 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 14:46:34.233393 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 14:46:34.234593 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 14:46:34.234593 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:46:34.234593 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:46:34.234593 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:46:34.234593 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:46:34.234593 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:46:34.234593 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:46:34.234593 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:46:34.234593 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application with Lock( Sep 01 14:46:34.234593 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:46:34.234593 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:46:34.234593 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:46:34.234593 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:46:34.234593 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:46:34.234593 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:46:34.234593 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:46:34.234593 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:46:34.235690 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 14:46:34.235690 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 14:46:34.235690 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 14:46:34.235690 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 14:46:34.235690 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 14:46:34.235690 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 14:46:34.235690 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 14:46:34.235690 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Sep 01 14:46:34.235690 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 413/808] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Sep 1 14:46:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:46:34.239793 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153ba97a-2523-412e-9ea2-ad89c803e091 None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:34.298037 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 396/809] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 1 14:46:34 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 14:46:34.298898 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f23d2e-352d-4d3c-9c85-59403b14eba5 None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:34.356760 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153ba97a-2523-412e-9ea2-ad89c803e091 None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:34.367729 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:34.414151 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7e3a94-0942-438f-b144-5a64b90444cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:34.415148 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7e3a94-0942-438f-b144-5a64b90444cd None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:34.415866 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7e3a94-0942-438f-b144-5a64b90444cd None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:34.416298 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7e3a94-0942-438f-b144-5a64b90444cd None admin] PATH_INFO: `/v3/users` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:34.417838 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7e3a94-0942-438f-b144-5a64b90444cd None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:46:34.418338 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7e3a94-0942-438f-b144-5a64b90444cd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:34.419657 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f23d2e-352d-4d3c-9c85-59403b14eba5 None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:34.420113 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7e3a94-0942-438f-b144-5a64b90444cd None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:34.444577 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 414/810] 10.222.0.33 () {56 vars in 1133 bytes} [Thu Sep 1 14:46:34 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:46:34.446300 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7e3a94-0942-438f-b144-5a64b90444cd None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:34.455989 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a88048-7130-49ee-bf3a-19737aa0a5a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:34.456724 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a88048-7130-49ee-bf3a-19737aa0a5a0 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:34.457098 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a88048-7130-49ee-bf3a-19737aa0a5a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:34.458396 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a88048-7130-49ee-bf3a-19737aa0a5a0 None admin] PATH_INFO: `/v3/users` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:34.458893 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a88048-7130-49ee-bf3a-19737aa0a5a0 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81341) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:46:34.459283 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a88048-7130-49ee-bf3a-19737aa0a5a0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:34.460938 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a88048-7130-49ee-bf3a-19737aa0a5a0 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:34.462564 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:34.479334 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 397/811] 10.222.0.33 () {56 vars in 1135 bytes} [Thu Sep 1 14:46:34 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:46:34.480448 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a88048-7130-49ee-bf3a-19737aa0a5a0 None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:34.558148 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7e3a94-0942-438f-b144-5a64b90444cd None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:34.591394 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a88048-7130-49ee-bf3a-19737aa0a5a0 None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:34.592750 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16a62fe-1346-46ea-801a-d5601ba577f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:34.593357 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16a62fe-1346-46ea-801a-d5601ba577f2 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:34.593638 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16a62fe-1346-46ea-801a-d5601ba577f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:34.593907 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16a62fe-1346-46ea-801a-d5601ba577f2 None admin] PATH_INFO: `/v3/projects` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:34.594634 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16a62fe-1346-46ea-801a-d5601ba577f2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:46:34.594939 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16a62fe-1346-46ea-801a-d5601ba577f2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:34.596357 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16a62fe-1346-46ea-801a-d5601ba577f2 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:34.605825 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:34.607707 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 415/812] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Sep 1 14:46:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:46:34.608523 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16a62fe-1346-46ea-801a-d5601ba577f2 None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:34.627964 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad22d626-7b3a-4f58-b5ff-a3530c221683 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:34.628569 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad22d626-7b3a-4f58-b5ff-a3530c221683 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:34.628849 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad22d626-7b3a-4f58-b5ff-a3530c221683 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:34.629177 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad22d626-7b3a-4f58-b5ff-a3530c221683 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:34.669994 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:34.701958 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad22d626-7b3a-4f58-b5ff-a3530c221683 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:34.703456 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad22d626-7b3a-4f58-b5ff-a3530c221683 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:34.709424 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16a62fe-1346-46ea-801a-d5601ba577f2 None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:34.741956 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ac47a5-95c3-4f2d-9266-bc2147329a9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:34.742678 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ac47a5-95c3-4f2d-9266-bc2147329a9e None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:34.743031 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ac47a5-95c3-4f2d-9266-bc2147329a9e None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:34.743371 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ac47a5-95c3-4f2d-9266-bc2147329a9e None admin] PATH_INFO: `/v3/services/network` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:34.744029 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ac47a5-95c3-4f2d-9266-bc2147329a9e None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:34.745456 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ac47a5-95c3-4f2d-9266-bc2147329a9e None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:34.754043 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application [None req-88ac47a5-95c3-4f2d-9266-bc2147329a9e None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 01 14:46:34.754043 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:46:34.754043 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 14:46:34.754043 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 14:46:34.754043 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 14:46:34.754043 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 14:46:34.754043 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 14:46:34.754043 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 14:46:34.754043 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 01 14:46:34.754043 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Sep 01 14:46:34.754043 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 14:46:34.754043 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Sep 01 14:46:34.754043 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:46:34.754043 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:46:34.754043 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:46:34.754043 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:46:34.754043 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:46:34.755786 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:46:34.755786 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:46:34.755786 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:46:34.755786 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:46:34.755786 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:46:34.755786 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:46:34.755786 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 14:46:34.755786 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 14:46:34.755786 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 14:46:34.755786 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 14:46:34.755786 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:46:34.755786 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:46:34.755786 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:46:34.755786 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:46:34.755786 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:46:34.755786 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:46:34.756665 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:46:34.756665 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application with Lock( Sep 01 14:46:34.756665 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:46:34.756665 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:46:34.756665 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:46:34.756665 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:46:34.756665 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:46:34.756665 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:46:34.756665 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:46:34.756665 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:46:34.756665 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 14:46:34.756665 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 14:46:34.756665 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 01 14:46:34.756665 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Sep 01 14:46:34.756665 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 416/813] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Sep 1 14:46:34 2022] GET /identity/v3/services/network => generated 88 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:46:34.757542 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ac47a5-95c3-4f2d-9266-bc2147329a9e None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:34.802793 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 398/814] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Sep 1 14:46:34 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 14:46:34.803781 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad22d626-7b3a-4f58-b5ff-a3530c221683 None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:34.818720 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:34.855274 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ac47a5-95c3-4f2d-9266-bc2147329a9e None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:34.880476 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:34.889514 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e3789a-182f-462e-938e-bbad891c119a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:34.890133 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e3789a-182f-462e-938e-bbad891c119a None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:34.890441 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e3789a-182f-462e-938e-bbad891c119a None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:34.890713 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e3789a-182f-462e-938e-bbad891c119a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:34.900629 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e3789a-182f-462e-938e-bbad891c119a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:34.901067 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad22d626-7b3a-4f58-b5ff-a3530c221683 None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:34.902062 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e3789a-182f-462e-938e-bbad891c119a None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:34.916808 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application [None req-13e3789a-182f-462e-938e-bbad891c119a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 14:46:34.916808 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:46:34.916808 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:46:34.916808 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:46:34.916808 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:46:34.916808 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:46:34.916808 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:46:34.916808 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:46:34.916808 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:46:34.916808 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:46:34.916808 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:46:34.916808 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:46:34.916808 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 14:46:34.916808 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 14:46:34.916808 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 14:46:34.916808 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 14:46:34.917947 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:46:34.917947 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:46:34.917947 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:46:34.917947 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:46:34.917947 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:46:34.917947 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:46:34.917947 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:46:34.917947 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application with Lock( Sep 01 14:46:34.917947 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:46:34.917947 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:46:34.917947 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:46:34.917947 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:46:34.917947 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:46:34.917947 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:46:34.917947 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:46:34.917947 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:46:34.917947 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 14:46:34.919031 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 14:46:34.919031 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 14:46:34.919031 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Sep 01 14:46:34.919031 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 417/815] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 1 14:46:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:46:34.919031 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e3789a-182f-462e-938e-bbad891c119a None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:34.939992 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7916470e-5262-47a9-a3fe-246650d12ca9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:34.940616 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7916470e-5262-47a9-a3fe-246650d12ca9 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:34.940898 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7916470e-5262-47a9-a3fe-246650d12ca9 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:34.941171 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7916470e-5262-47a9-a3fe-246650d12ca9 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:34.941750 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7916470e-5262-47a9-a3fe-246650d12ca9 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:34.943165 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7916470e-5262-47a9-a3fe-246650d12ca9 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:34.953827 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application [None req-7916470e-5262-47a9-a3fe-246650d12ca9 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 01 14:46:34.953827 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:46:34.953827 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 14:46:34.953827 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 14:46:34.953827 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 14:46:34.953827 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 14:46:34.953827 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 14:46:34.953827 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 14:46:34.953827 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 01 14:46:34.953827 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Sep 01 14:46:34.953827 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 14:46:34.953827 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Sep 01 14:46:34.953827 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:46:34.953827 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:46:34.953827 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:46:34.953827 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:46:34.953827 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:46:34.955013 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:46:34.955013 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:46:34.955013 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:46:34.955013 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:46:34.955013 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:46:34.955013 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:46:34.955013 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 14:46:34.955013 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 14:46:34.955013 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 14:46:34.955013 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 14:46:34.955013 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:46:34.955013 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:46:34.955013 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:46:34.955013 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:46:34.955013 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:46:34.955013 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:46:34.955948 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:46:34.955948 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application with Lock( Sep 01 14:46:34.955948 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:46:34.955948 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:46:34.955948 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:46:34.955948 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:46:34.955948 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:46:34.955948 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:46:34.955948 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:46:34.955948 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:46:34.955948 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 14:46:34.955948 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 14:46:34.955948 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 01 14:46:34.955948 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Sep 01 14:46:34.955948 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 399/816] 10.222.0.33 () {56 vars in 1129 bytes} [Thu Sep 1 14:46:34 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:46:34.956797 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7916470e-5262-47a9-a3fe-246650d12ca9 None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:35.018336 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e3789a-182f-462e-938e-bbad891c119a None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:35.034124 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:35.052526 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbda296-9c4b-4072-85a8-5e35bc1213bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:35.053178 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbda296-9c4b-4072-85a8-5e35bc1213bf None admin] REQUEST_METHOD: `PUT` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:35.053454 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbda296-9c4b-4072-85a8-5e35bc1213bf None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:35.053730 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbda296-9c4b-4072-85a8-5e35bc1213bf None admin] PATH_INFO: `/v3/projects/e8c85b4bd17e4e89b56bf0f831b474d5/users/24b6f275ffb04ffeae3c2097bfd040f5/roles/5e3f992e804046f381b3de8bf2a512e2` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:35.056335 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7916470e-5262-47a9-a3fe-246650d12ca9 None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:35.067705 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbda296-9c4b-4072-85a8-5e35bc1213bf None admin] RBAC: Authorizing `identity:create_grant(project_id=e8c85b4bd17e4e89b56bf0f831b474d5, user_id=24b6f275ffb04ffeae3c2097bfd040f5, role_id=5e3f992e804046f381b3de8bf2a512e2)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:35.069137 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbda296-9c4b-4072-85a8-5e35bc1213bf None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:35.081187 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 418/817] 10.222.0.33 () {58 vars in 1439 bytes} [Thu Sep 1 14:46:34 2022] PUT /identity/v3/projects/e8c85b4bd17e4e89b56bf0f831b474d5/users/24b6f275ffb04ffeae3c2097bfd040f5/roles/5e3f992e804046f381b3de8bf2a512e2 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:46:35.081971 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbda296-9c4b-4072-85a8-5e35bc1213bf None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:35.089920 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ae049f-9f16-45af-8a14-fa89abb4b72e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:35.090595 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ae049f-9f16-45af-8a14-fa89abb4b72e None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:35.091010 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ae049f-9f16-45af-8a14-fa89abb4b72e None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:35.091284 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ae049f-9f16-45af-8a14-fa89abb4b72e None admin] PATH_INFO: `/v3/services` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:35.091945 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ae049f-9f16-45af-8a14-fa89abb4b72e None admin] RBAC: Adding query filter params (name=network) {{(pid=81341) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:46:35.092249 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ae049f-9f16-45af-8a14-fa89abb4b72e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:35.093834 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ae049f-9f16-45af-8a14-fa89abb4b72e None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:35.099793 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 400/818] 10.222.0.33 () {56 vars in 1112 bytes} [Thu Sep 1 14:46:34 2022] GET /identity/v3/services?name=network => generated 124 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:46:35.100660 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ae049f-9f16-45af-8a14-fa89abb4b72e None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:35.119955 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:35.208495 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbda296-9c4b-4072-85a8-5e35bc1213bf None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:35.251992 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304cf7d1-d868-47a8-8f7e-c3f5e19aa6ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:35.252628 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304cf7d1-d868-47a8-8f7e-c3f5e19aa6ff None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:35.252904 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304cf7d1-d868-47a8-8f7e-c3f5e19aa6ff None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:35.253172 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304cf7d1-d868-47a8-8f7e-c3f5e19aa6ff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:35.263617 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304cf7d1-d868-47a8-8f7e-c3f5e19aa6ff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:35.265070 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304cf7d1-d868-47a8-8f7e-c3f5e19aa6ff None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:35.280285 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application [None req-304cf7d1-d868-47a8-8f7e-c3f5e19aa6ff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 14:46:35.280285 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:46:35.280285 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:46:35.280285 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:46:35.280285 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:46:35.280285 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:46:35.280285 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:46:35.280285 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:46:35.280285 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:46:35.280285 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:46:35.280285 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:46:35.280285 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:46:35.280285 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 14:46:35.280285 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 14:46:35.280285 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 14:46:35.280285 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 14:46:35.281278 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:46:35.281278 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:46:35.281278 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:46:35.281278 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:46:35.281278 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:46:35.281278 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:46:35.281278 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:46:35.281278 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application with Lock( Sep 01 14:46:35.281278 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:46:35.281278 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:46:35.281278 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:46:35.281278 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:46:35.281278 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:46:35.281278 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:46:35.281278 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:46:35.281278 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:46:35.281278 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 14:46:35.282199 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 14:46:35.282199 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 14:46:35.282199 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Sep 01 14:46:35.282199 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 419/819] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 1 14:46:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:46:35.282497 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304cf7d1-d868-47a8-8f7e-c3f5e19aa6ff None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:35.341775 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ae049f-9f16-45af-8a14-fa89abb4b72e None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:35.377939 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e3cf26-2903-417d-8d5d-5ddc183b5807 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:35.378145 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:35.378745 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e3cf26-2903-417d-8d5d-5ddc183b5807 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:35.379111 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e3cf26-2903-417d-8d5d-5ddc183b5807 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:35.379417 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e3cf26-2903-417d-8d5d-5ddc183b5807 None admin] PATH_INFO: `/v3/domains` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:35.380270 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e3cf26-2903-417d-8d5d-5ddc183b5807 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81341) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:46:35.380574 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e3cf26-2903-417d-8d5d-5ddc183b5807 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:35.381979 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e3cf26-2903-417d-8d5d-5ddc183b5807 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:35.394279 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 401/820] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Sep 1 14:46:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:46:35.395621 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e3cf26-2903-417d-8d5d-5ddc183b5807 None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:35.398263 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:35.406777 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304cf7d1-d868-47a8-8f7e-c3f5e19aa6ff None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:35.441628 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa76d48d-ccf8-4942-bf38-660146bc8e23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:35.442252 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa76d48d-ccf8-4942-bf38-660146bc8e23 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:35.442531 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa76d48d-ccf8-4942-bf38-660146bc8e23 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:35.442798 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa76d48d-ccf8-4942-bf38-660146bc8e23 None admin] PATH_INFO: `/v3/services` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:35.443423 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa76d48d-ccf8-4942-bf38-660146bc8e23 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:46:35.443719 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa76d48d-ccf8-4942-bf38-660146bc8e23 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:35.445077 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa76d48d-ccf8-4942-bf38-660146bc8e23 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:35.449986 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 420/821] 10.222.0.33 () {56 vars in 1124 bytes} [Thu Sep 1 14:46:35 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:46:35.451009 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa76d48d-ccf8-4942-bf38-660146bc8e23 None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:35.505397 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e3cf26-2903-417d-8d5d-5ddc183b5807 None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:35.539883 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc51a2a-e7ab-4575-b029-2675fd640c8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:35.540497 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc51a2a-e7ab-4575-b029-2675fd640c8a None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:35.540775 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc51a2a-e7ab-4575-b029-2675fd640c8a None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:35.541042 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc51a2a-e7ab-4575-b029-2675fd640c8a None admin] PATH_INFO: `/v3/services` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:35.541666 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc51a2a-e7ab-4575-b029-2675fd640c8a None admin] RBAC: Adding query filter params (type=network) {{(pid=81341) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:46:35.541964 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc51a2a-e7ab-4575-b029-2675fd640c8a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:35.543386 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc51a2a-e7ab-4575-b029-2675fd640c8a None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:35.554631 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 402/822] 10.222.0.33 () {56 vars in 1112 bytes} [Thu Sep 1 14:46:35 2022] GET /identity/v3/services?type=network => generated 350 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:46:35.555466 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc51a2a-e7ab-4575-b029-2675fd640c8a None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:35.559650 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa76d48d-ccf8-4942-bf38-660146bc8e23 None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:35.594994 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa778f25-a5ba-42a7-8b37-1a61b916b036 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:35.595603 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa778f25-a5ba-42a7-8b37-1a61b916b036 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:35.595878 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa778f25-a5ba-42a7-8b37-1a61b916b036 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:35.596147 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa778f25-a5ba-42a7-8b37-1a61b916b036 None admin] PATH_INFO: `/v3/domains` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:35.596783 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa778f25-a5ba-42a7-8b37-1a61b916b036 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:46:35.597087 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa778f25-a5ba-42a7-8b37-1a61b916b036 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:35.598505 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa778f25-a5ba-42a7-8b37-1a61b916b036 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:35.610172 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 421/823] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Sep 1 14:46:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:46:35.610905 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa778f25-a5ba-42a7-8b37-1a61b916b036 None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:35.614270 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:35.631898 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:35.672837 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc51a2a-e7ab-4575-b029-2675fd640c8a None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:35.708268 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a1eb35-aef9-4fa7-8da5-667c4d3964bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:35.708899 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a1eb35-aef9-4fa7-8da5-667c4d3964bc None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:35.709177 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a1eb35-aef9-4fa7-8da5-667c4d3964bc None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:35.709488 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a1eb35-aef9-4fa7-8da5-667c4d3964bc None admin] PATH_INFO: `/v3/projects/service` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:35.716306 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a1eb35-aef9-4fa7-8da5-667c4d3964bc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:35.717751 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a1eb35-aef9-4fa7-8da5-667c4d3964bc None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:35.724198 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa778f25-a5ba-42a7-8b37-1a61b916b036 None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:35.728870 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application [None req-94a1eb35-aef9-4fa7-8da5-667c4d3964bc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 14:46:35.728870 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:46:35.728870 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:46:35.728870 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:46:35.728870 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:46:35.728870 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:46:35.728870 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:46:35.728870 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:46:35.728870 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:46:35.728870 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:46:35.728870 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:46:35.728870 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:46:35.728870 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 14:46:35.728870 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 14:46:35.728870 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 14:46:35.729948 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 14:46:35.729948 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:46:35.729948 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:46:35.729948 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:46:35.729948 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:46:35.729948 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:46:35.729948 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:46:35.729948 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:46:35.729948 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application with Lock( Sep 01 14:46:35.729948 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:46:35.729948 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:46:35.729948 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:46:35.729948 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:46:35.729948 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:46:35.729948 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:46:35.729948 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:46:35.729948 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:46:35.731096 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 14:46:35.731096 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 14:46:35.731096 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 14:46:35.731096 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 14:46:35.731096 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 14:46:35.731096 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 14:46:35.731096 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 14:46:35.731096 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Sep 01 14:46:35.737048 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 403/824] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Sep 1 14:46:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:46:35.737936 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a1eb35-aef9-4fa7-8da5-667c4d3964bc None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:35.762023 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3aa84ad-cc9e-4aaf-8b26-8dea62d45559 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:35.762707 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3aa84ad-cc9e-4aaf-8b26-8dea62d45559 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:35.763079 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3aa84ad-cc9e-4aaf-8b26-8dea62d45559 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:35.763439 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3aa84ad-cc9e-4aaf-8b26-8dea62d45559 None admin] PATH_INFO: `/v3/services` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:35.764160 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3aa84ad-cc9e-4aaf-8b26-8dea62d45559 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:46:35.764576 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3aa84ad-cc9e-4aaf-8b26-8dea62d45559 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:35.766021 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3aa84ad-cc9e-4aaf-8b26-8dea62d45559 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:35.771419 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 422/825] 10.222.0.33 () {56 vars in 1124 bytes} [Thu Sep 1 14:46:35 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:46:35.776679 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3aa84ad-cc9e-4aaf-8b26-8dea62d45559 None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:35.843778 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a1eb35-aef9-4fa7-8da5-667c4d3964bc None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:35.847615 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:35.870122 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:35.879164 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af59b165-c132-4325-96de-5917cccec345 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:35.879767 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af59b165-c132-4325-96de-5917cccec345 None admin] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:35.880046 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af59b165-c132-4325-96de-5917cccec345 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:35.880321 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af59b165-c132-4325-96de-5917cccec345 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:35.881007 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af59b165-c132-4325-96de-5917cccec345 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:35.882426 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af59b165-c132-4325-96de-5917cccec345 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:35.887144 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3aa84ad-cc9e-4aaf-8b26-8dea62d45559 None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:35.903698 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 404/826] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Sep 1 14:46:35 2022] POST /identity/v3/endpoints => generated 337 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:46:35.911820 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af59b165-c132-4325-96de-5917cccec345 None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:35.940704 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f820f4-fc9b-4aee-857c-39903c504fcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:35.941359 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f820f4-fc9b-4aee-857c-39903c504fcc None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:35.941729 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f820f4-fc9b-4aee-857c-39903c504fcc None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:35.942005 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f820f4-fc9b-4aee-857c-39903c504fcc None admin] PATH_INFO: `/v3/projects/service` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:35.952173 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f820f4-fc9b-4aee-857c-39903c504fcc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:35.953927 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f820f4-fc9b-4aee-857c-39903c504fcc None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:35.971129 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application [None req-68f820f4-fc9b-4aee-857c-39903c504fcc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 14:46:35.971129 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:46:35.971129 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:46:35.971129 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:46:35.971129 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:46:35.971129 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:46:35.971129 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:46:35.971129 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:46:35.971129 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:46:35.971129 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:46:35.971129 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:46:35.971129 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:46:35.971129 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 14:46:35.971129 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 14:46:35.971129 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 14:46:35.972132 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 14:46:35.972132 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:46:35.972132 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:46:35.972132 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:46:35.972132 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:46:35.972132 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:46:35.972132 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:46:35.972132 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:46:35.972132 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application with Lock( Sep 01 14:46:35.972132 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:46:35.972132 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:46:35.972132 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:46:35.972132 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:46:35.972132 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:46:35.972132 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:46:35.972132 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:46:35.972132 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:46:35.973691 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 14:46:35.973691 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 14:46:35.973691 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 14:46:35.973691 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 14:46:35.973691 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 14:46:35.973691 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 14:46:35.973691 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 14:46:35.973691 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Sep 01 14:46:35.973691 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 423/827] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Sep 1 14:46:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:46:35.974331 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f820f4-fc9b-4aee-857c-39903c504fcc None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:36.057715 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af59b165-c132-4325-96de-5917cccec345 None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:36.102757 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f8a77a-2002-4805-9e9c-e5dc0f9461d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:36.103654 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f8a77a-2002-4805-9e9c-e5dc0f9461d0 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:36.104396 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f8a77a-2002-4805-9e9c-e5dc0f9461d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:36.104687 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f8a77a-2002-4805-9e9c-e5dc0f9461d0 None admin] PATH_INFO: `/v3/projects` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:36.105672 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f8a77a-2002-4805-9e9c-e5dc0f9461d0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81341) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:46:36.105981 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f8a77a-2002-4805-9e9c-e5dc0f9461d0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:36.107170 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f820f4-fc9b-4aee-857c-39903c504fcc None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:36.107480 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f8a77a-2002-4805-9e9c-e5dc0f9461d0 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:36.128466 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 405/828] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Sep 1 14:46:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:46:36.128466 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f8a77a-2002-4805-9e9c-e5dc0f9461d0 None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:36.138526 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:36.157615 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495bd915-dccf-43de-8d90-dee90a1de1ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:36.158962 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495bd915-dccf-43de-8d90-dee90a1de1ab None admin] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:36.159451 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495bd915-dccf-43de-8d90-dee90a1de1ab None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:36.160056 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495bd915-dccf-43de-8d90-dee90a1de1ab None admin] PATH_INFO: `/v3/endpoints` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:36.162178 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495bd915-dccf-43de-8d90-dee90a1de1ab None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:36.163963 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495bd915-dccf-43de-8d90-dee90a1de1ab None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:36.178270 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:36.191151 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 424/829] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Sep 1 14:46:35 2022] POST /identity/v3/endpoints => generated 346 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:46:36.191809 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495bd915-dccf-43de-8d90-dee90a1de1ab None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:36.271585 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f8a77a-2002-4805-9e9c-e5dc0f9461d0 None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:36.322084 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495bd915-dccf-43de-8d90-dee90a1de1ab None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:36.328644 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061ec1ef-0f1a-4389-bef1-8c26d41e76cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:36.329249 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061ec1ef-0f1a-4389-bef1-8c26d41e76cf None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:36.329566 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061ec1ef-0f1a-4389-bef1-8c26d41e76cf None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:36.329841 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061ec1ef-0f1a-4389-bef1-8c26d41e76cf None admin] PATH_INFO: `/v3/projects` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:36.330559 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061ec1ef-0f1a-4389-bef1-8c26d41e76cf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81341) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:46:36.330857 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061ec1ef-0f1a-4389-bef1-8c26d41e76cf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:36.333423 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061ec1ef-0f1a-4389-bef1-8c26d41e76cf None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:36.354627 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 406/830] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Sep 1 14:46:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:46:36.355500 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061ec1ef-0f1a-4389-bef1-8c26d41e76cf None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:36.364947 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc878a6b-842d-4632-9ade-3d4484772931 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:36.365617 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc878a6b-842d-4632-9ade-3d4484772931 None admin] REQUEST_METHOD: `PUT` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:36.365898 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc878a6b-842d-4632-9ade-3d4484772931 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:36.366174 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc878a6b-842d-4632-9ade-3d4484772931 None admin] PATH_INFO: `/v3/projects/e8c85b4bd17e4e89b56bf0f831b474d5/users/73a5bda5a6ea46e9863ccafd57dee339/roles/2a2adb769e55476b82e5014897e0a9c0` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:36.368146 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc878a6b-842d-4632-9ade-3d4484772931 None admin] RBAC: Authorizing `identity:create_grant(project_id=e8c85b4bd17e4e89b56bf0f831b474d5, user_id=73a5bda5a6ea46e9863ccafd57dee339, role_id=2a2adb769e55476b82e5014897e0a9c0)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:36.369580 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc878a6b-842d-4632-9ade-3d4484772931 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:36.387715 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 425/831] 10.222.0.33 () {58 vars in 1439 bytes} [Thu Sep 1 14:46:36 2022] PUT /identity/v3/projects/e8c85b4bd17e4e89b56bf0f831b474d5/users/73a5bda5a6ea46e9863ccafd57dee339/roles/2a2adb769e55476b82e5014897e0a9c0 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:46:36.478968 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:36.483869 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061ec1ef-0f1a-4389-bef1-8c26d41e76cf None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:36.524351 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8affe638-e3a9-4a37-bbe6-68da64aa900e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:36.525218 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8affe638-e3a9-4a37-bbe6-68da64aa900e None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:36.525667 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8affe638-e3a9-4a37-bbe6-68da64aa900e None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:36.526076 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8affe638-e3a9-4a37-bbe6-68da64aa900e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:36.526980 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8affe638-e3a9-4a37-bbe6-68da64aa900e None admin] RBAC: Adding query filter params (role.id=2a2adb769e55476b82e5014897e0a9c0, scope.project.id=e8c85b4bd17e4e89b56bf0f831b474d5, user.id=4858bb8621854ec8aba836e7d9050498) {{(pid=81341) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:46:36.527478 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8affe638-e3a9-4a37-bbe6-68da64aa900e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:36.529130 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8affe638-e3a9-4a37-bbe6-68da64aa900e None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:36.536563 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 407/832] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Sep 1 14:46:36 2022] GET /identity/v3/role_assignments?user.id=4858bb8621854ec8aba836e7d9050498&scope.project.id=e8c85b4bd17e4e89b56bf0f831b474d5&role.id=2a2adb769e55476b82e5014897e0a9c0 => generated 608 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:46:37.042903 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231e47e2-82c3-4618-8b2f-1bf6a92fea2d None None] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:37.043094 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231e47e2-82c3-4618-8b2f-1bf6a92fea2d None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:37.043360 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231e47e2-82c3-4618-8b2f-1bf6a92fea2d None None] PATH_INFO: `/` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:37.044923 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 426/833] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 1 14:46:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:46:37.050303 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f66ab95-2ece-40f7-b7a4-6c8769c52197 None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:37.050742 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f66ab95-2ece-40f7-b7a4-6c8769c52197 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:37.050940 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f66ab95-2ece-40f7-b7a4-6c8769c52197 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:37.069760 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-4f66ab95-2ece-40f7-b7a4-6c8769c52197 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:46:37.071248 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-4f66ab95-2ece-40f7-b7a4-6c8769c52197 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:46:37.141123 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 408/834] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 1 14:46:37 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 14:46:37.415201 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231e47e2-82c3-4618-8b2f-1bf6a92fea2d None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:37.454601 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:37.526982 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231e47e2-82c3-4618-8b2f-1bf6a92fea2d None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:37.562121 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eab864a-5839-4c7f-bfa9-e2d3819780fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:37.562783 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eab864a-5839-4c7f-bfa9-e2d3819780fa None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:37.563055 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eab864a-5839-4c7f-bfa9-e2d3819780fa None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:37.563319 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eab864a-5839-4c7f-bfa9-e2d3819780fa None admin] PATH_INFO: `/v3/roles/service` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:37.574029 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eab864a-5839-4c7f-bfa9-e2d3819780fa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 14:46:37.578882 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application [None req-6eab864a-5839-4c7f-bfa9-e2d3819780fa None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 14:46:37.578882 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:46:37.578882 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:46:37.578882 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:46:37.578882 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:46:37.578882 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:46:37.578882 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:46:37.578882 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:46:37.578882 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:46:37.578882 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:46:37.578882 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:46:37.578882 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:46:37.578882 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 14:46:37.578882 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 14:46:37.578882 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 14:46:37.578882 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 14:46:37.579824 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 14:46:37.579824 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 14:46:37.579824 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 14:46:37.579824 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 14:46:37.579824 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:46:37.579824 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:46:37.579824 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:46:37.579824 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:46:37.579824 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:46:37.579824 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:46:37.579824 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:46:37.579824 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application with Lock( Sep 01 14:46:37.579824 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:46:37.579824 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:46:37.579824 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:46:37.579824 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:46:37.580735 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:46:37.580735 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:46:37.580735 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:46:37.580735 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:46:37.580735 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 14:46:37.580735 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 14:46:37.580735 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 14:46:37.580735 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 14:46:37.580735 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 14:46:37.580735 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 14:46:37.580735 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 14:46:37.580735 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Sep 01 14:46:37.580735 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 427/835] 10.222.0.33 () {56 vars in 1102 bytes} [Thu Sep 1 14:46:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:46:37.586673 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f66ab95-2ece-40f7-b7a4-6c8769c52197 None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:37.684188 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:37.694566 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f66ab95-2ece-40f7-b7a4-6c8769c52197 None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:37.732584 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d7392b-77b5-4e39-9f64-1d8e02be1f38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:37.733195 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d7392b-77b5-4e39-9f64-1d8e02be1f38 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:37.733481 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d7392b-77b5-4e39-9f64-1d8e02be1f38 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:37.733752 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d7392b-77b5-4e39-9f64-1d8e02be1f38 None admin] PATH_INFO: `/v3/roles` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:37.734503 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d7392b-77b5-4e39-9f64-1d8e02be1f38 None admin] RBAC: Adding query filter params (name=service) {{(pid=81341) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:46:37.734809 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d7392b-77b5-4e39-9f64-1d8e02be1f38 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:37.736198 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d7392b-77b5-4e39-9f64-1d8e02be1f38 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:37.745114 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 409/836] 10.222.0.33 () {56 vars in 1103 bytes} [Thu Sep 1 14:46:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:46:37.751316 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eab864a-5839-4c7f-bfa9-e2d3819780fa None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:37.851430 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:37.857850 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eab864a-5839-4c7f-bfa9-e2d3819780fa None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:37.893836 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385edeb4-eb9f-4dd5-909c-0767fead713e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:37.894499 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385edeb4-eb9f-4dd5-909c-0767fead713e None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:37.895325 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385edeb4-eb9f-4dd5-909c-0767fead713e None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:37.895639 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385edeb4-eb9f-4dd5-909c-0767fead713e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:37.906097 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385edeb4-eb9f-4dd5-909c-0767fead713e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:37.907606 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385edeb4-eb9f-4dd5-909c-0767fead713e None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:37.927109 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application [None req-385edeb4-eb9f-4dd5-909c-0767fead713e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 14:46:37.927109 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:46:37.927109 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:46:37.927109 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:46:37.927109 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:46:37.927109 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:46:37.927109 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:46:37.927109 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:46:37.927109 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:46:37.927109 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:46:37.927109 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:46:37.927109 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:46:37.927109 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 14:46:37.927109 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 14:46:37.927109 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 14:46:37.927109 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 14:46:37.928132 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:46:37.928132 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:46:37.928132 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:46:37.928132 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:46:37.928132 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:46:37.928132 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:46:37.928132 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:46:37.928132 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application with Lock( Sep 01 14:46:37.928132 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:46:37.928132 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:46:37.928132 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:46:37.928132 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:46:37.928132 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:46:37.928132 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:46:37.928132 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:46:37.928132 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:46:37.928132 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 14:46:37.929040 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 14:46:37.929040 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 14:46:37.929040 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Sep 01 14:46:37.929040 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 428/837] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 1 14:46:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:46:37.934046 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d7392b-77b5-4e39-9f64-1d8e02be1f38 None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:38.040823 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d7392b-77b5-4e39-9f64-1d8e02be1f38 None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:38.077023 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463120d5-2436-48a0-b412-7d9787cbfd35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:38.077804 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463120d5-2436-48a0-b412-7d9787cbfd35 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:38.077968 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463120d5-2436-48a0-b412-7d9787cbfd35 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:38.078226 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463120d5-2436-48a0-b412-7d9787cbfd35 None admin] PATH_INFO: `/v3/domains` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:38.078906 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463120d5-2436-48a0-b412-7d9787cbfd35 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81341) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:46:38.079210 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463120d5-2436-48a0-b412-7d9787cbfd35 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:38.080605 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463120d5-2436-48a0-b412-7d9787cbfd35 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:38.094029 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 410/838] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Sep 1 14:46:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:46:38.099704 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385edeb4-eb9f-4dd5-909c-0767fead713e None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:38.203464 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385edeb4-eb9f-4dd5-909c-0767fead713e None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:38.233458 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81284b8c-eb7e-4ffc-8707-e87fff9ea892 None None] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:38.234084 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81284b8c-eb7e-4ffc-8707-e87fff9ea892 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:38.234624 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81284b8c-eb7e-4ffc-8707-e87fff9ea892 None None] PATH_INFO: `/` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:38.236514 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 411/839] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 1 14:46:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:46:38.240350 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb54fd9-6af6-4219-bc70-0470233f90a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:38.240977 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb54fd9-6af6-4219-bc70-0470233f90a1 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:38.241317 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb54fd9-6af6-4219-bc70-0470233f90a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:38.241615 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb54fd9-6af6-4219-bc70-0470233f90a1 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:38.243144 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adb00aa-48d1-4668-aa47-7dd307de1046 None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:38.243431 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adb00aa-48d1-4668-aa47-7dd307de1046 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:38.243702 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adb00aa-48d1-4668-aa47-7dd307de1046 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:38.249888 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb54fd9-6af6-4219-bc70-0470233f90a1 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:38.251346 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb54fd9-6af6-4219-bc70-0470233f90a1 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:38.263289 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application [None req-1eb54fd9-6af6-4219-bc70-0470233f90a1 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 01 14:46:38.263289 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:46:38.263289 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:46:38.263289 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:46:38.263289 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:46:38.263289 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:46:38.263289 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:46:38.263289 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:46:38.263289 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:46:38.263289 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:46:38.263289 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:46:38.263289 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:46:38.263289 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 14:46:38.263289 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 14:46:38.263289 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 14:46:38.263289 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 14:46:38.264347 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:46:38.264347 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:46:38.264347 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 14:46:38.264347 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 14:46:38.264347 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 14:46:38.264347 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 14:46:38.264347 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:46:38.264347 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:46:38.264347 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:46:38.264347 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:46:38.264347 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:46:38.264347 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application with Lock( Sep 01 14:46:38.264347 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:46:38.264347 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:46:38.264347 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:46:38.264347 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:46:38.264347 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:46:38.264347 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:46:38.265310 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:46:38.265310 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:46:38.265310 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 14:46:38.265310 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 14:46:38.265310 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 14:46:38.265310 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 14:46:38.265310 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 14:46:38.265310 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 14:46:38.265310 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 01 14:46:38.265310 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Sep 01 14:46:38.265858 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-0adb00aa-48d1-4668-aa47-7dd307de1046 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:46:38.265933 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 429/840] 10.222.0.33 () {56 vars in 1117 bytes} [Thu Sep 1 14:46:38 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:46:38.266009 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-0adb00aa-48d1-4668-aa47-7dd307de1046 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:46:38.278489 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb54fd9-6af6-4219-bc70-0470233f90a1 None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:38.279830 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:38.293588 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:38.344523 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 412/841] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 1 14:46:38 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 14:46:38.391166 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb54fd9-6af6-4219-bc70-0470233f90a1 None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:38.403273 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a208a7-a569-4c88-929f-b0550a2d4ede None None] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:38.403556 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a208a7-a569-4c88-929f-b0550a2d4ede None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:38.403832 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a208a7-a569-4c88-929f-b0550a2d4ede None None] PATH_INFO: `/` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:38.405647 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 413/842] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 1 14:46:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:46:38.410581 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e4b76d-ead6-42c4-a841-c51ef071b7c5 None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:38.410865 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e4b76d-ead6-42c4-a841-c51ef071b7c5 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:38.411135 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e4b76d-ead6-42c4-a841-c51ef071b7c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:38.427328 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3586a04-896f-49d4-b91b-37d9e5af2d8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:38.427954 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3586a04-896f-49d4-b91b-37d9e5af2d8f None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:38.428180 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3586a04-896f-49d4-b91b-37d9e5af2d8f None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:38.428444 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3586a04-896f-49d4-b91b-37d9e5af2d8f None admin] PATH_INFO: `/v3/users` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:38.429166 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3586a04-896f-49d4-b91b-37d9e5af2d8f None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:46:38.429612 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3586a04-896f-49d4-b91b-37d9e5af2d8f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:38.430464 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-28e4b76d-ead6-42c4-a841-c51ef071b7c5 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:46:38.431158 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3586a04-896f-49d4-b91b-37d9e5af2d8f None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:38.431666 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-28e4b76d-ead6-42c4-a841-c51ef071b7c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:46:38.450506 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 430/843] 10.222.0.33 () {56 vars in 1149 bytes} [Thu Sep 1 14:46:38 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:46:38.456104 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3586a04-896f-49d4-b91b-37d9e5af2d8f None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:38.526406 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 414/844] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 1 14:46:38 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 14:46:38.533013 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0852b655-6c66-4906-95e3-20beec360bf1 None None] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:38.533295 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0852b655-6c66-4906-95e3-20beec360bf1 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:38.533655 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:38.534361 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0852b655-6c66-4906-95e3-20beec360bf1 None None] PATH_INFO: `/` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:38.535955 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 415/845] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 1 14:46:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 14:46:38.540843 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3133dc7-7cd2-40a9-ab37-26c6058cdfa8 None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:38.541176 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3133dc7-7cd2-40a9-ab37-26c6058cdfa8 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:38.541450 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3133dc7-7cd2-40a9-ab37-26c6058cdfa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:38.562153 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-b3133dc7-7cd2-40a9-ab37-26c6058cdfa8 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:46:38.563925 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-b3133dc7-7cd2-40a9-ab37-26c6058cdfa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:46:38.569635 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3586a04-896f-49d4-b91b-37d9e5af2d8f None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:38.604740 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72c4705-2f49-4690-8924-81359ceba5a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:38.605339 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72c4705-2f49-4690-8924-81359ceba5a4 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:38.605612 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72c4705-2f49-4690-8924-81359ceba5a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:38.605874 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72c4705-2f49-4690-8924-81359ceba5a4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:38.642275 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 416/846] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 1 14:46:38 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 14:46:38.643049 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3133dc7-7cd2-40a9-ab37-26c6058cdfa8 None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:38.646960 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:38.693034 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72c4705-2f49-4690-8924-81359ceba5a4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:38.694510 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72c4705-2f49-4690-8924-81359ceba5a4 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:38.756330 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3133dc7-7cd2-40a9-ab37-26c6058cdfa8 None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:38.763289 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:38.792980 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4426855-c9d6-4635-8ecf-4169e30852da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:38.793805 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4426855-c9d6-4635-8ecf-4169e30852da None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:38.794191 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4426855-c9d6-4635-8ecf-4169e30852da None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:38.794587 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4426855-c9d6-4635-8ecf-4169e30852da None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:38.795270 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4426855-c9d6-4635-8ecf-4169e30852da None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:38.796756 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4426855-c9d6-4635-8ecf-4169e30852da None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:38.803703 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 431/847] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Sep 1 14:46:38 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 14:46:38.807481 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72c4705-2f49-4690-8924-81359ceba5a4 None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:38.810931 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application [None req-e4426855-c9d6-4635-8ecf-4169e30852da None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 01 14:46:38.810931 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:46:38.810931 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 14:46:38.810931 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 14:46:38.810931 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 14:46:38.810931 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 14:46:38.810931 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 14:46:38.810931 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 14:46:38.810931 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 01 14:46:38.810931 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Sep 01 14:46:38.810931 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 14:46:38.810931 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Sep 01 14:46:38.810931 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:46:38.810931 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:46:38.810931 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:46:38.810931 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:46:38.810931 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:46:38.812123 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:46:38.812123 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:46:38.812123 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:46:38.812123 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:46:38.812123 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:46:38.812123 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:46:38.812123 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 14:46:38.812123 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 14:46:38.812123 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 14:46:38.812123 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 14:46:38.812123 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:46:38.812123 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:46:38.812123 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:46:38.812123 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:46:38.812123 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:46:38.812123 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:46:38.813204 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:46:38.813204 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application with Lock( Sep 01 14:46:38.813204 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:46:38.813204 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:46:38.813204 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:46:38.813204 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:46:38.813204 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:46:38.813204 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:46:38.813204 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:46:38.813204 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:46:38.813204 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 14:46:38.813204 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 14:46:38.813204 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 01 14:46:38.813204 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Sep 01 14:46:38.813204 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 417/848] 10.222.0.33 () {56 vars in 1114 bytes} [Thu Sep 1 14:46:38 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:46:38.814182 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4426855-c9d6-4635-8ecf-4169e30852da None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:38.883653 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:38.915334 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72c4705-2f49-4690-8924-81359ceba5a4 None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:38.918615 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4426855-c9d6-4635-8ecf-4169e30852da None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:38.949445 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de4fa90-7f12-464d-a80f-cc49392f1b2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:38.950183 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de4fa90-7f12-464d-a80f-cc49392f1b2b None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:38.950486 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de4fa90-7f12-464d-a80f-cc49392f1b2b None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:38.950769 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de4fa90-7f12-464d-a80f-cc49392f1b2b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:38.951856 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348955f1-1eda-4b94-b5af-88b982ee6aad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:38.952478 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348955f1-1eda-4b94-b5af-88b982ee6aad None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:38.952758 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348955f1-1eda-4b94-b5af-88b982ee6aad None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:38.953039 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348955f1-1eda-4b94-b5af-88b982ee6aad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:38.961381 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de4fa90-7f12-464d-a80f-cc49392f1b2b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 14:46:38.964078 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348955f1-1eda-4b94-b5af-88b982ee6aad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:38.965524 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348955f1-1eda-4b94-b5af-88b982ee6aad None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:38.966127 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application [None req-9de4fa90-7f12-464d-a80f-cc49392f1b2b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 14:46:38.966127 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:46:38.966127 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:46:38.966127 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:46:38.966127 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:46:38.966127 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:46:38.966127 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:46:38.966127 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:46:38.966127 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:46:38.966127 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:46:38.966127 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:46:38.966127 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:46:38.966127 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 14:46:38.966127 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 14:46:38.966127 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 14:46:38.966127 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 14:46:38.967243 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 14:46:38.967243 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 14:46:38.967243 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 14:46:38.967243 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 14:46:38.967243 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:46:38.967243 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:46:38.967243 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:46:38.967243 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:46:38.967243 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:46:38.967243 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:46:38.967243 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:46:38.967243 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application with Lock( Sep 01 14:46:38.967243 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:46:38.967243 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:46:38.967243 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:46:38.967243 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:46:38.968586 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:46:38.968586 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:46:38.968586 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:46:38.968586 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:46:38.968586 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 14:46:38.968586 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 14:46:38.968586 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 14:46:38.968586 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 14:46:38.968586 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 14:46:38.968586 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 14:46:38.968586 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 14:46:38.968586 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Sep 01 14:46:38.968586 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 432/849] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Sep 1 14:46:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 14:46:38.970216 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de4fa90-7f12-464d-a80f-cc49392f1b2b None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:38.980759 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application [None req-348955f1-1eda-4b94-b5af-88b982ee6aad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 14:46:38.980759 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:46:38.980759 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:46:38.980759 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:46:38.980759 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:46:38.980759 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:46:38.980759 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:46:38.980759 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:46:38.980759 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:46:38.980759 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:46:38.980759 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:46:38.980759 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:46:38.980759 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 14:46:38.980759 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 14:46:38.980759 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 14:46:38.980759 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 14:46:38.981892 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:46:38.981892 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:46:38.981892 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:46:38.981892 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:46:38.981892 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:46:38.981892 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:46:38.981892 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:46:38.981892 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application with Lock( Sep 01 14:46:38.981892 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:46:38.981892 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:46:38.981892 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:46:38.981892 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:46:38.981892 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:46:38.981892 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:46:38.981892 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:46:38.981892 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:46:38.981892 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 14:46:38.982999 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 14:46:38.982999 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 14:46:38.982999 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Sep 01 14:46:38.986282 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 418/850] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 1 14:46:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:46:38.987091 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348955f1-1eda-4b94-b5af-88b982ee6aad None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:38.997674 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:39.070516 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de4fa90-7f12-464d-a80f-cc49392f1b2b None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:39.085648 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348955f1-1eda-4b94-b5af-88b982ee6aad None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:39.104614 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad0b4e0-f64a-4f42-81c3-15354356728c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:39.105211 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad0b4e0-f64a-4f42-81c3-15354356728c None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:39.105475 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad0b4e0-f64a-4f42-81c3-15354356728c None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:39.105713 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:39.105825 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad0b4e0-f64a-4f42-81c3-15354356728c None admin] PATH_INFO: `/v3/services` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:39.106417 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad0b4e0-f64a-4f42-81c3-15354356728c None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:46:39.106791 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad0b4e0-f64a-4f42-81c3-15354356728c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:39.108289 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad0b4e0-f64a-4f42-81c3-15354356728c None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:39.112544 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 433/851] 10.222.0.33 () {56 vars in 1114 bytes} [Thu Sep 1 14:46:38 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:46:39.113322 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad0b4e0-f64a-4f42-81c3-15354356728c None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:39.120139 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66702312-db13-42e0-8c60-bc49c037b5a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:39.120764 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66702312-db13-42e0-8c60-bc49c037b5a6 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:39.121042 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66702312-db13-42e0-8c60-bc49c037b5a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:39.121427 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66702312-db13-42e0-8c60-bc49c037b5a6 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:39.122011 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66702312-db13-42e0-8c60-bc49c037b5a6 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:39.123438 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66702312-db13-42e0-8c60-bc49c037b5a6 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:39.132120 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application [None req-66702312-db13-42e0-8c60-bc49c037b5a6 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 01 14:46:39.132120 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:46:39.132120 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 14:46:39.132120 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 14:46:39.132120 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 14:46:39.132120 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 14:46:39.132120 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 14:46:39.132120 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 14:46:39.132120 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 01 14:46:39.132120 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Sep 01 14:46:39.132120 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 14:46:39.132120 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Sep 01 14:46:39.132120 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:46:39.132120 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:46:39.132120 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:46:39.132120 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:46:39.133206 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:46:39.133206 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:46:39.133206 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:46:39.133206 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:46:39.133206 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:46:39.133206 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:46:39.133206 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:46:39.133206 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 14:46:39.133206 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 14:46:39.133206 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 14:46:39.133206 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 14:46:39.133206 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:46:39.133206 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:46:39.133206 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:46:39.133206 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:46:39.133206 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:46:39.133206 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:46:39.134136 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:46:39.134136 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application with Lock( Sep 01 14:46:39.134136 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:46:39.134136 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:46:39.134136 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:46:39.134136 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:46:39.134136 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:46:39.134136 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:46:39.134136 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:46:39.134136 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:46:39.134136 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 14:46:39.134136 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 14:46:39.134136 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 01 14:46:39.134136 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Sep 01 14:46:39.134136 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 419/852] 10.222.0.33 () {56 vars in 1132 bytes} [Thu Sep 1 14:46:38 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 146 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:46:39.135009 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66702312-db13-42e0-8c60-bc49c037b5a6 None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:39.202060 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:39.212540 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad0b4e0-f64a-4f42-81c3-15354356728c None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:39.238401 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66702312-db13-42e0-8c60-bc49c037b5a6 None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:39.246159 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf15f0b-919c-4e96-ad79-09707f24de01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:39.246790 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf15f0b-919c-4e96-ad79-09707f24de01 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:39.247069 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf15f0b-919c-4e96-ad79-09707f24de01 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:39.247338 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf15f0b-919c-4e96-ad79-09707f24de01 None admin] PATH_INFO: `/v3/roles` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:39.247997 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf15f0b-919c-4e96-ad79-09707f24de01 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:46:39.248304 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf15f0b-919c-4e96-ad79-09707f24de01 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:39.249659 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf15f0b-919c-4e96-ad79-09707f24de01 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:39.257490 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 434/853] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Sep 1 14:46:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:46:39.258279 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf15f0b-919c-4e96-ad79-09707f24de01 None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:39.271573 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f767facb-651c-4835-b188-47e5a9bc9659 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:39.272170 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f767facb-651c-4835-b188-47e5a9bc9659 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:39.272453 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f767facb-651c-4835-b188-47e5a9bc9659 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:39.272721 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f767facb-651c-4835-b188-47e5a9bc9659 None admin] PATH_INFO: `/v3/domains` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:39.273357 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f767facb-651c-4835-b188-47e5a9bc9659 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81341) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:46:39.273653 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f767facb-651c-4835-b188-47e5a9bc9659 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:39.275078 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f767facb-651c-4835-b188-47e5a9bc9659 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:39.287097 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 420/854] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Sep 1 14:46:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:46:39.288046 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f767facb-651c-4835-b188-47e5a9bc9659 None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:39.333454 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:39.361925 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf15f0b-919c-4e96-ad79-09707f24de01 None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:39.387620 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f767facb-651c-4835-b188-47e5a9bc9659 None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:39.394490 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4847493f-fe73-49df-bfd0-2ca400484575 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:39.395621 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4847493f-fe73-49df-bfd0-2ca400484575 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:39.395894 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4847493f-fe73-49df-bfd0-2ca400484575 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:39.396158 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4847493f-fe73-49df-bfd0-2ca400484575 None admin] PATH_INFO: `/v3/services` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:39.396777 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4847493f-fe73-49df-bfd0-2ca400484575 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:46:39.397070 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4847493f-fe73-49df-bfd0-2ca400484575 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:39.398440 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4847493f-fe73-49df-bfd0-2ca400484575 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:39.402705 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 435/855] 10.222.0.33 () {56 vars in 1114 bytes} [Thu Sep 1 14:46:39 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:46:39.403629 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4847493f-fe73-49df-bfd0-2ca400484575 None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:39.422886 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba1e3b7-bec8-486a-95c5-59c5d3c73cff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:39.423525 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba1e3b7-bec8-486a-95c5-59c5d3c73cff None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:39.423904 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba1e3b7-bec8-486a-95c5-59c5d3c73cff None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:39.424182 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba1e3b7-bec8-486a-95c5-59c5d3c73cff None admin] PATH_INFO: `/v3/services` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:39.424837 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba1e3b7-bec8-486a-95c5-59c5d3c73cff None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81341) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:46:39.425142 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba1e3b7-bec8-486a-95c5-59c5d3c73cff None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:39.426661 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba1e3b7-bec8-486a-95c5-59c5d3c73cff None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:39.432810 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 421/856] 10.222.0.33 () {56 vars in 1126 bytes} [Thu Sep 1 14:46:39 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:46:39.433789 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba1e3b7-bec8-486a-95c5-59c5d3c73cff None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:39.439478 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:39.538400 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4847493f-fe73-49df-bfd0-2ca400484575 None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:39.547788 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba1e3b7-bec8-486a-95c5-59c5d3c73cff None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:39.558354 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:39.577074 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e29d47e-c93f-4092-adb4-5bc200c7a6f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:39.577708 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e29d47e-c93f-4092-adb4-5bc200c7a6f8 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:39.577992 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e29d47e-c93f-4092-adb4-5bc200c7a6f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:39.578284 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e29d47e-c93f-4092-adb4-5bc200c7a6f8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:39.590037 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e29d47e-c93f-4092-adb4-5bc200c7a6f8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:39.591791 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e29d47e-c93f-4092-adb4-5bc200c7a6f8 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:39.596419 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b0edc2-41db-4ac3-b170-2abb1a93ddb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:39.597055 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b0edc2-41db-4ac3-b170-2abb1a93ddb3 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:39.597397 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b0edc2-41db-4ac3-b170-2abb1a93ddb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:39.597704 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b0edc2-41db-4ac3-b170-2abb1a93ddb3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:39.605087 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b0edc2-41db-4ac3-b170-2abb1a93ddb3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:39.606354 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application [None req-0e29d47e-c93f-4092-adb4-5bc200c7a6f8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 14:46:39.606354 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:46:39.606354 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:46:39.606354 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:46:39.606354 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:46:39.606354 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:46:39.606354 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:46:39.606354 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:46:39.606354 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:46:39.606354 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:46:39.606354 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:46:39.606354 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:46:39.606354 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 14:46:39.606354 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 14:46:39.606354 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 14:46:39.606354 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 14:46:39.607326 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:46:39.607326 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:46:39.607326 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:46:39.607326 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:46:39.607326 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:46:39.607326 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:46:39.607326 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:46:39.607326 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application with Lock( Sep 01 14:46:39.607326 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:46:39.607326 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:46:39.607326 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:46:39.607326 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:46:39.607326 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:46:39.607326 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:46:39.607326 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:46:39.607326 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:46:39.607326 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 14:46:39.608330 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 14:46:39.608330 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 14:46:39.608330 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Sep 01 14:46:39.608509 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b0edc2-41db-4ac3-b170-2abb1a93ddb3 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:39.608590 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 436/857] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 1 14:46:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:46:39.608590 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e29d47e-c93f-4092-adb4-5bc200c7a6f8 None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:39.620547 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application [None req-43b0edc2-41db-4ac3-b170-2abb1a93ddb3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 14:46:39.620547 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:46:39.620547 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:46:39.620547 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:46:39.620547 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:46:39.620547 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:46:39.620547 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:46:39.620547 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:46:39.620547 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:46:39.620547 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:46:39.620547 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:46:39.620547 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:46:39.620547 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 14:46:39.620547 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 14:46:39.620547 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 14:46:39.621542 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 14:46:39.621542 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:46:39.621542 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:46:39.621542 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:46:39.621542 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:46:39.621542 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:46:39.621542 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:46:39.621542 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:46:39.621542 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application with Lock( Sep 01 14:46:39.621542 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:46:39.621542 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:46:39.621542 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:46:39.621542 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:46:39.621542 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:46:39.621542 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:46:39.621542 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:46:39.621542 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:46:39.622548 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 14:46:39.622548 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 14:46:39.622548 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 14:46:39.622548 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 14:46:39.622548 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 14:46:39.622548 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 14:46:39.622548 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 14:46:39.622548 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Sep 01 14:46:39.626427 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 422/858] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Sep 1 14:46:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:46:39.627082 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b0edc2-41db-4ac3-b170-2abb1a93ddb3 None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:39.685045 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:39.720178 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e29d47e-c93f-4092-adb4-5bc200c7a6f8 None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:39.744588 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b0edc2-41db-4ac3-b170-2abb1a93ddb3 None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:39.756223 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1982b324-2447-4dc2-baf2-5263c315815c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:39.756830 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1982b324-2447-4dc2-baf2-5263c315815c None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:39.757165 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1982b324-2447-4dc2-baf2-5263c315815c None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:39.757376 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1982b324-2447-4dc2-baf2-5263c315815c None admin] PATH_INFO: `/v3/services` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:39.758010 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1982b324-2447-4dc2-baf2-5263c315815c None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:46:39.758331 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1982b324-2447-4dc2-baf2-5263c315815c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:39.759740 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1982b324-2447-4dc2-baf2-5263c315815c None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:39.770281 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 437/859] 10.222.0.33 () {56 vars in 1126 bytes} [Thu Sep 1 14:46:39 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:46:39.770281 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1982b324-2447-4dc2-baf2-5263c315815c None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:39.783058 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e747711-f5d2-47a8-80d1-be1c25a233fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:39.783291 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e747711-f5d2-47a8-80d1-be1c25a233fa None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:39.783375 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e747711-f5d2-47a8-80d1-be1c25a233fa None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:39.783640 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e747711-f5d2-47a8-80d1-be1c25a233fa None admin] PATH_INFO: `/v3/domains` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:39.784303 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e747711-f5d2-47a8-80d1-be1c25a233fa None admin] RBAC: Adding query filter params (name=Default) {{(pid=81341) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:46:39.784607 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e747711-f5d2-47a8-80d1-be1c25a233fa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:39.786082 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e747711-f5d2-47a8-80d1-be1c25a233fa None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:39.802475 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 423/860] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Sep 1 14:46:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:46:39.807947 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e747711-f5d2-47a8-80d1-be1c25a233fa None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:39.843594 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:39.914086 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1982b324-2447-4dc2-baf2-5263c315815c None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:39.940014 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e747711-f5d2-47a8-80d1-be1c25a233fa None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:39.955324 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:39.959003 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bc737c-1495-4530-9770-bbc0503e32e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:39.959783 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bc737c-1495-4530-9770-bbc0503e32e7 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:39.960171 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bc737c-1495-4530-9770-bbc0503e32e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:39.960442 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bc737c-1495-4530-9770-bbc0503e32e7 None admin] PATH_INFO: `/v3/projects` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:39.961143 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bc737c-1495-4530-9770-bbc0503e32e7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:46:39.961451 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bc737c-1495-4530-9770-bbc0503e32e7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:39.963529 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bc737c-1495-4530-9770-bbc0503e32e7 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:39.984616 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 438/861] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Sep 1 14:46:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:46:39.986987 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438af039-ea6a-49c4-99a1-e42d9bd5e076 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:39.987623 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438af039-ea6a-49c4-99a1-e42d9bd5e076 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:39.987901 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438af039-ea6a-49c4-99a1-e42d9bd5e076 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:39.988171 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438af039-ea6a-49c4-99a1-e42d9bd5e076 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:39.990891 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bc737c-1495-4530-9770-bbc0503e32e7 None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:40.005968 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438af039-ea6a-49c4-99a1-e42d9bd5e076 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:40.006931 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438af039-ea6a-49c4-99a1-e42d9bd5e076 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:40.019436 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application [None req-438af039-ea6a-49c4-99a1-e42d9bd5e076 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 01 14:46:40.019436 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:46:40.019436 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:46:40.019436 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:46:40.019436 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:46:40.019436 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:46:40.019436 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:46:40.019436 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:46:40.019436 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:46:40.019436 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:46:40.019436 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:46:40.019436 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:46:40.019436 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 14:46:40.019436 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 14:46:40.019436 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 14:46:40.019436 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 14:46:40.020492 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:46:40.020492 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:46:40.020492 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 14:46:40.020492 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 14:46:40.020492 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 14:46:40.020492 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 14:46:40.020492 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:46:40.020492 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:46:40.020492 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:46:40.020492 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:46:40.020492 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:46:40.020492 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application with Lock( Sep 01 14:46:40.020492 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:46:40.020492 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:46:40.020492 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:46:40.020492 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:46:40.020492 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:46:40.020492 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:46:40.021587 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:46:40.021587 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:46:40.021587 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 14:46:40.021587 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 14:46:40.021587 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 14:46:40.021587 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 14:46:40.021587 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 14:46:40.021587 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 14:46:40.021587 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 01 14:46:40.021587 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Sep 01 14:46:40.021587 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 424/862] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Sep 1 14:46:39 2022] GET /identity/v3/users/swift => generated 83 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:46:40.028694 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438af039-ea6a-49c4-99a1-e42d9bd5e076 None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:40.124419 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bc737c-1495-4530-9770-bbc0503e32e7 None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:40.130598 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:40.146172 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438af039-ea6a-49c4-99a1-e42d9bd5e076 None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:40.167043 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb5f74a-5b14-43d1-b4f8-6a7f15d2c5a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:40.167649 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb5f74a-5b14-43d1-b4f8-6a7f15d2c5a8 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:40.167929 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb5f74a-5b14-43d1-b4f8-6a7f15d2c5a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:40.168198 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb5f74a-5b14-43d1-b4f8-6a7f15d2c5a8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:40.168925 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb5f74a-5b14-43d1-b4f8-6a7f15d2c5a8 None admin] RBAC: Adding query filter params (role.id=5e3f992e804046f381b3de8bf2a512e2, scope.project.id=e8c85b4bd17e4e89b56bf0f831b474d5, user.id=24b6f275ffb04ffeae3c2097bfd040f5) {{(pid=81342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:46:40.169232 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb5f74a-5b14-43d1-b4f8-6a7f15d2c5a8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:40.171406 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb5f74a-5b14-43d1-b4f8-6a7f15d2c5a8 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:40.180017 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 439/863] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Sep 1 14:46:39 2022] GET /identity/v3/role_assignments?user.id=24b6f275ffb04ffeae3c2097bfd040f5&scope.project.id=e8c85b4bd17e4e89b56bf0f831b474d5&role.id=5e3f992e804046f381b3de8bf2a512e2 => generated 608 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:46:40.194196 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfc9ccb-dcba-4455-8372-5464158de44c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:40.195559 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfc9ccb-dcba-4455-8372-5464158de44c None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:40.196038 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfc9ccb-dcba-4455-8372-5464158de44c None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:40.196507 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfc9ccb-dcba-4455-8372-5464158de44c None admin] PATH_INFO: `/v3/users` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:40.197462 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfc9ccb-dcba-4455-8372-5464158de44c None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81341) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:46:40.197970 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfc9ccb-dcba-4455-8372-5464158de44c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:40.199719 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfc9ccb-dcba-4455-8372-5464158de44c None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:40.220468 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 425/864] 10.222.0.33 () {56 vars in 1135 bytes} [Thu Sep 1 14:46:40 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:46:40.227630 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb5f74a-5b14-43d1-b4f8-6a7f15d2c5a8 None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:40.291436 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:40.340136 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb5f74a-5b14-43d1-b4f8-6a7f15d2c5a8 None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:40.377175 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f4b700-169d-4e17-8d47-5d0acf885845 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:40.378123 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f4b700-169d-4e17-8d47-5d0acf885845 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:40.378623 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f4b700-169d-4e17-8d47-5d0acf885845 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:40.379087 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f4b700-169d-4e17-8d47-5d0acf885845 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:40.451243 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f4b700-169d-4e17-8d47-5d0acf885845 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:40.452707 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f4b700-169d-4e17-8d47-5d0acf885845 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:40.511058 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:40.557546 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 440/865] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Sep 1 14:46:40 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 14:46:40.563363 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfc9ccb-dcba-4455-8372-5464158de44c None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:40.677372 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfc9ccb-dcba-4455-8372-5464158de44c None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:40.712712 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266e0755-9d95-49a9-b535-3ef7e207a684 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:40.713342 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266e0755-9d95-49a9-b535-3ef7e207a684 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:40.713622 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266e0755-9d95-49a9-b535-3ef7e207a684 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:40.713893 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266e0755-9d95-49a9-b535-3ef7e207a684 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:40.724545 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266e0755-9d95-49a9-b535-3ef7e207a684 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:40.725995 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266e0755-9d95-49a9-b535-3ef7e207a684 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:40.732792 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:40.741437 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application [None req-266e0755-9d95-49a9-b535-3ef7e207a684 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 14:46:40.741437 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:46:40.741437 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:46:40.741437 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:46:40.741437 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:46:40.741437 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:46:40.741437 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:46:40.741437 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:46:40.741437 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:46:40.741437 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:46:40.741437 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:46:40.741437 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:46:40.741437 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 14:46:40.741437 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 14:46:40.741437 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 14:46:40.741437 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 14:46:40.742450 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:46:40.742450 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:46:40.742450 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:46:40.742450 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:46:40.742450 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:46:40.742450 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:46:40.742450 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:46:40.742450 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application with Lock( Sep 01 14:46:40.742450 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:46:40.742450 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:46:40.742450 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:46:40.742450 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:46:40.742450 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:46:40.742450 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:46:40.742450 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:46:40.742450 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:46:40.742450 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 14:46:40.743385 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 14:46:40.743385 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 14:46:40.743385 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Sep 01 14:46:40.743385 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 426/866] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 1 14:46:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 14:46:40.748502 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f4b700-169d-4e17-8d47-5d0acf885845 None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:40.850602 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f4b700-169d-4e17-8d47-5d0acf885845 None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:40.894626 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5f1e1f-6e7c-4bc8-95f8-90043a9f1dc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:40.895741 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5f1e1f-6e7c-4bc8-95f8-90043a9f1dc0 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:40.896024 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5f1e1f-6e7c-4bc8-95f8-90043a9f1dc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:40.896295 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5f1e1f-6e7c-4bc8-95f8-90043a9f1dc0 None admin] PATH_INFO: `/v3/domains` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:40.896954 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5f1e1f-6e7c-4bc8-95f8-90043a9f1dc0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:46:40.897252 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5f1e1f-6e7c-4bc8-95f8-90043a9f1dc0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:40.898827 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5f1e1f-6e7c-4bc8-95f8-90043a9f1dc0 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:40.911305 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 441/867] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Sep 1 14:46:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:46:40.916918 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266e0755-9d95-49a9-b535-3ef7e207a684 None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:41.027080 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266e0755-9d95-49a9-b535-3ef7e207a684 None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:41.061450 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf135fe-d90e-4775-95f2-1bfbd2edc7fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:41.062130 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf135fe-d90e-4775-95f2-1bfbd2edc7fc None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:41.062442 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf135fe-d90e-4775-95f2-1bfbd2edc7fc None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:41.062717 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf135fe-d90e-4775-95f2-1bfbd2edc7fc None admin] PATH_INFO: `/v3/projects/service` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:41.069978 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf135fe-d90e-4775-95f2-1bfbd2edc7fc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:41.071735 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf135fe-d90e-4775-95f2-1bfbd2edc7fc None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:41.083354 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application [None req-3cf135fe-d90e-4775-95f2-1bfbd2edc7fc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 14:46:41.083354 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:46:41.083354 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:46:41.083354 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:46:41.083354 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:46:41.083354 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:46:41.083354 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:46:41.083354 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:46:41.083354 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:46:41.083354 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:46:41.083354 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:46:41.083354 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:46:41.083354 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 14:46:41.083354 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 14:46:41.083354 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 14:46:41.084265 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 14:46:41.084265 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:46:41.084265 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:46:41.084265 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:46:41.084265 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:46:41.084265 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:46:41.084265 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:46:41.084265 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:46:41.084265 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application with Lock( Sep 01 14:46:41.084265 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:46:41.084265 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:46:41.084265 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:46:41.084265 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:46:41.084265 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:46:41.084265 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:46:41.084265 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:46:41.084265 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:46:41.085181 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 14:46:41.085181 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 14:46:41.085181 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 14:46:41.085181 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 14:46:41.085181 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 14:46:41.085181 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 14:46:41.085181 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 14:46:41.085181 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Sep 01 14:46:41.085181 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 427/868] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Sep 1 14:46:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:46:41.091931 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5f1e1f-6e7c-4bc8-95f8-90043a9f1dc0 None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:41.111180 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:41.202418 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5f1e1f-6e7c-4bc8-95f8-90043a9f1dc0 None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:41.237645 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7f659d-a49e-41f5-87dc-030c8c31b54f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:41.238805 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7f659d-a49e-41f5-87dc-030c8c31b54f None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:41.239085 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7f659d-a49e-41f5-87dc-030c8c31b54f None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:41.239351 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7f659d-a49e-41f5-87dc-030c8c31b54f None admin] PATH_INFO: `/v3/projects` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:41.240097 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7f659d-a49e-41f5-87dc-030c8c31b54f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:46:41.240396 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7f659d-a49e-41f5-87dc-030c8c31b54f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:41.241916 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7f659d-a49e-41f5-87dc-030c8c31b54f None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:41.255563 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 442/869] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Sep 1 14:46:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:46:41.261355 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf135fe-d90e-4775-95f2-1bfbd2edc7fc None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:41.369561 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:41.416379 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf135fe-d90e-4775-95f2-1bfbd2edc7fc None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:41.452229 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda76e01-f49c-49c4-8a55-1a1eea5d272a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:41.453264 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda76e01-f49c-49c4-8a55-1a1eea5d272a None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:41.453860 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda76e01-f49c-49c4-8a55-1a1eea5d272a None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:41.454376 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda76e01-f49c-49c4-8a55-1a1eea5d272a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:41.455310 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda76e01-f49c-49c4-8a55-1a1eea5d272a None admin] RBAC: Adding query filter params (role.id=2a2adb769e55476b82e5014897e0a9c0, scope.project.id=e8c85b4bd17e4e89b56bf0f831b474d5, user.id=73a5bda5a6ea46e9863ccafd57dee339) {{(pid=81341) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:46:41.455824 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda76e01-f49c-49c4-8a55-1a1eea5d272a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:41.457539 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda76e01-f49c-49c4-8a55-1a1eea5d272a None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:41.464827 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 428/870] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Sep 1 14:46:41 2022] GET /identity/v3/role_assignments?user.id=73a5bda5a6ea46e9863ccafd57dee339&scope.project.id=e8c85b4bd17e4e89b56bf0f831b474d5&role.id=2a2adb769e55476b82e5014897e0a9c0 => generated 608 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:46:41.479327 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0caefa59-493a-4299-8e1f-6bb94093d449 None None] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:41.479833 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0caefa59-493a-4299-8e1f-6bb94093d449 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:41.479910 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0caefa59-493a-4299-8e1f-6bb94093d449 None None] PATH_INFO: `/` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:41.485510 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 443/871] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 1 14:46:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:46:41.487553 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc46f5a5-3e07-4783-ad24-c41e16b019d6 None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:41.488087 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc46f5a5-3e07-4783-ad24-c41e16b019d6 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:41.489091 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc46f5a5-3e07-4783-ad24-c41e16b019d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:41.510256 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-cc46f5a5-3e07-4783-ad24-c41e16b019d6 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:46:41.512267 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-cc46f5a5-3e07-4783-ad24-c41e16b019d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:46:41.642629 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 429/872] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 1 14:46:41 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 14:46:41.879703 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94498ccd-c209-4845-ac4a-f821b275b7b5 None None] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:41.880099 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94498ccd-c209-4845-ac4a-f821b275b7b5 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:41.880291 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94498ccd-c209-4845-ac4a-f821b275b7b5 None None] PATH_INFO: `/` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:41.881837 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 444/873] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 1 14:46:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:46:41.887048 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83b1274-5f51-43d4-b15c-fd2f37c3e185 None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:41.887335 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83b1274-5f51-43d4-b15c-fd2f37c3e185 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:41.887609 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83b1274-5f51-43d4-b15c-fd2f37c3e185 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:41.905726 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-b83b1274-5f51-43d4-b15c-fd2f37c3e185 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:46:41.906970 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-b83b1274-5f51-43d4-b15c-fd2f37c3e185 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:46:41.933689 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94498ccd-c209-4845-ac4a-f821b275b7b5 None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:41.957999 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:41.990730 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 430/874] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 1 14:46:41 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 14:46:42.031685 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:42.057794 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94498ccd-c209-4845-ac4a-f821b275b7b5 None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:42.094107 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9eecb38-f49c-437f-8a6a-8cca94f00f3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:42.095007 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9eecb38-f49c-437f-8a6a-8cca94f00f3a None admin] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:42.095435 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9eecb38-f49c-437f-8a6a-8cca94f00f3a None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:42.095855 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9eecb38-f49c-437f-8a6a-8cca94f00f3a None admin] PATH_INFO: `/v3/services` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:42.096654 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9eecb38-f49c-437f-8a6a-8cca94f00f3a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:42.098769 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9eecb38-f49c-437f-8a6a-8cca94f00f3a None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:42.117950 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 445/875] 10.222.0.33 () {60 vars in 1141 bytes} [Thu Sep 1 14:46:41 2022] POST /identity/v3/services => generated 251 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:46:42.269059 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83b1274-5f51-43d4-b15c-fd2f37c3e185 None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:42.372938 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83b1274-5f51-43d4-b15c-fd2f37c3e185 None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:42.385561 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecb5ff1-ea6d-4dfe-9fef-8c26fcff7996 None None] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:42.385855 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecb5ff1-ea6d-4dfe-9fef-8c26fcff7996 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:42.386113 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecb5ff1-ea6d-4dfe-9fef-8c26fcff7996 None None] PATH_INFO: `/` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:42.387762 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 446/876] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 1 14:46:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:46:42.392800 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab2bf2b-883c-4e84-8834-1e701e9fa39e None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:42.393087 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab2bf2b-883c-4e84-8834-1e701e9fa39e None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:42.393544 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab2bf2b-883c-4e84-8834-1e701e9fa39e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:42.412051 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-cab2bf2b-883c-4e84-8834-1e701e9fa39e None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:46:42.413246 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-cab2bf2b-883c-4e84-8834-1e701e9fa39e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:46:42.423734 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167df1c9-5782-4f3a-af55-baa8211e866a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:42.425038 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167df1c9-5782-4f3a-af55-baa8211e866a None admin] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:42.425822 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167df1c9-5782-4f3a-af55-baa8211e866a None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:42.426373 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167df1c9-5782-4f3a-af55-baa8211e866a None admin] PATH_INFO: `/v3/services` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:42.427504 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167df1c9-5782-4f3a-af55-baa8211e866a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:42.429872 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167df1c9-5782-4f3a-af55-baa8211e866a None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:42.451260 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 431/877] 10.222.0.33 () {60 vars in 1141 bytes} [Thu Sep 1 14:46:42 2022] POST /identity/v3/services => generated 269 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:46:42.503051 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 447/878] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 1 14:46:42 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 14:46:42.772867 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167df1c9-5782-4f3a-af55-baa8211e866a None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:42.813959 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:42.880753 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167df1c9-5782-4f3a-af55-baa8211e866a None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:42.914807 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2dacc1e-ddc4-4911-9ccf-a262b8c0b569 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:42.915371 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dacc1e-ddc4-4911-9ccf-a262b8c0b569 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:42.915653 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dacc1e-ddc4-4911-9ccf-a262b8c0b569 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:42.915928 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dacc1e-ddc4-4911-9ccf-a262b8c0b569 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:42.927242 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2dacc1e-ddc4-4911-9ccf-a262b8c0b569 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 14:46:42.932493 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application [None req-b2dacc1e-ddc4-4911-9ccf-a262b8c0b569 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 01 14:46:42.932493 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:46:42.932493 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:46:42.932493 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:46:42.932493 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:46:42.932493 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:46:42.932493 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:46:42.932493 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:46:42.932493 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:46:42.932493 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:46:42.932493 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:46:42.932493 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:46:42.932493 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 14:46:42.932493 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 14:46:42.932493 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 14:46:42.932493 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 14:46:42.933447 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 14:46:42.933447 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 14:46:42.933447 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 14:46:42.933447 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 14:46:42.933447 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:46:42.933447 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:46:42.933447 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:46:42.933447 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:46:42.933447 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:46:42.933447 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:46:42.933447 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:46:42.933447 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application with Lock( Sep 01 14:46:42.933447 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:46:42.933447 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:46:42.933447 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:46:42.933447 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:46:42.934339 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:46:42.934339 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:46:42.934339 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:46:42.934339 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:46:42.934339 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 14:46:42.934339 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 14:46:42.934339 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 14:46:42.934339 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 14:46:42.934339 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 14:46:42.934339 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 14:46:42.934339 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 01 14:46:42.934339 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Sep 01 14:46:42.934339 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 432/879] 10.222.0.33 () {56 vars in 1120 bytes} [Thu Sep 1 14:46:42 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:46:42.939549 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab2bf2b-883c-4e84-8834-1e701e9fa39e None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:43.004709 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:43.045543 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab2bf2b-883c-4e84-8834-1e701e9fa39e None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:43.081124 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d88435f-922e-45be-a277-d743393df82f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:43.081804 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d88435f-922e-45be-a277-d743393df82f None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:43.082127 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d88435f-922e-45be-a277-d743393df82f None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:43.082427 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d88435f-922e-45be-a277-d743393df82f None admin] PATH_INFO: `/v3/roles` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:43.083105 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d88435f-922e-45be-a277-d743393df82f None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:46:43.083446 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d88435f-922e-45be-a277-d743393df82f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:43.084784 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d88435f-922e-45be-a277-d743393df82f None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:43.093694 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 448/880] 10.222.0.33 () {56 vars in 1115 bytes} [Thu Sep 1 14:46:42 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:46:43.103337 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2dacc1e-ddc4-4911-9ccf-a262b8c0b569 None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:43.205530 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:43.209332 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2dacc1e-ddc4-4911-9ccf-a262b8c0b569 None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:43.244750 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35057acd-c4e9-4d21-9e5d-097a8fd64de6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:43.245389 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35057acd-c4e9-4d21-9e5d-097a8fd64de6 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:43.245668 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35057acd-c4e9-4d21-9e5d-097a8fd64de6 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:43.245943 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35057acd-c4e9-4d21-9e5d-097a8fd64de6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:43.258286 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35057acd-c4e9-4d21-9e5d-097a8fd64de6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:43.259827 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35057acd-c4e9-4d21-9e5d-097a8fd64de6 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:43.274990 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application [None req-35057acd-c4e9-4d21-9e5d-097a8fd64de6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 14:46:43.274990 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:46:43.274990 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:46:43.274990 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:46:43.274990 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:46:43.274990 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:46:43.274990 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:46:43.274990 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:46:43.274990 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:46:43.274990 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:46:43.274990 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:46:43.274990 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:46:43.274990 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 14:46:43.274990 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 14:46:43.274990 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 14:46:43.274990 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 14:46:43.276632 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:46:43.276632 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:46:43.276632 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:46:43.276632 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:46:43.276632 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:46:43.276632 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:46:43.276632 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:46:43.276632 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application with Lock( Sep 01 14:46:43.276632 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:46:43.276632 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:46:43.276632 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:46:43.276632 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:46:43.276632 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:46:43.276632 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:46:43.276632 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:46:43.276632 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:46:43.276632 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 14:46:43.277885 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 14:46:43.277885 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 14:46:43.277885 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Sep 01 14:46:43.277885 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 433/881] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 1 14:46:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:46:43.281997 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d88435f-922e-45be-a277-d743393df82f None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:43.385823 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d88435f-922e-45be-a277-d743393df82f None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:43.409349 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:43.421040 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3847deb-43dd-43e9-999f-ad82824e7ea4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:43.421640 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3847deb-43dd-43e9-999f-ad82824e7ea4 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:43.421918 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3847deb-43dd-43e9-999f-ad82824e7ea4 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:43.422190 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3847deb-43dd-43e9-999f-ad82824e7ea4 None admin] PATH_INFO: `/v3/domains` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:43.422862 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3847deb-43dd-43e9-999f-ad82824e7ea4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:46:43.423163 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3847deb-43dd-43e9-999f-ad82824e7ea4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:43.424532 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3847deb-43dd-43e9-999f-ad82824e7ea4 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:43.437468 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 449/882] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Sep 1 14:46:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:46:43.443061 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35057acd-c4e9-4d21-9e5d-097a8fd64de6 None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:43.546329 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35057acd-c4e9-4d21-9e5d-097a8fd64de6 None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:43.580561 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f90641-1144-4368-bfea-19c5ce919b44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:43.581211 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f90641-1144-4368-bfea-19c5ce919b44 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:43.581491 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f90641-1144-4368-bfea-19c5ce919b44 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:43.581765 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f90641-1144-4368-bfea-19c5ce919b44 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:43.591417 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f90641-1144-4368-bfea-19c5ce919b44 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:43.592935 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f90641-1144-4368-bfea-19c5ce919b44 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:43.602508 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0104f2-7971-4d3d-8d7f-3c1dd85c60b5 None None] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:43.602988 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0104f2-7971-4d3d-8d7f-3c1dd85c60b5 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:43.603093 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0104f2-7971-4d3d-8d7f-3c1dd85c60b5 None None] PATH_INFO: `/` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:43.604585 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 450/883] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 1 14:46:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 14:46:43.606039 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application [None req-73f90641-1144-4368-bfea-19c5ce919b44 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 01 14:46:43.606039 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:46:43.606039 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:46:43.606039 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:46:43.606039 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:46:43.606039 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:46:43.606039 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:46:43.606039 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:46:43.606039 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:46:43.606039 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:46:43.606039 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:46:43.606039 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:46:43.606039 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 14:46:43.606039 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 14:46:43.606039 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 14:46:43.606039 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 14:46:43.607005 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:46:43.607005 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:46:43.607005 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 14:46:43.607005 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 14:46:43.607005 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 14:46:43.607005 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 14:46:43.607005 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:46:43.607005 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:46:43.607005 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:46:43.607005 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:46:43.607005 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:46:43.607005 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application with Lock( Sep 01 14:46:43.607005 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:46:43.607005 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:46:43.607005 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:46:43.607005 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:46:43.607005 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:46:43.607005 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:46:43.608672 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:46:43.608672 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:46:43.608672 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 14:46:43.608672 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 14:46:43.608672 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 14:46:43.608672 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 14:46:43.608672 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 14:46:43.608672 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 14:46:43.608672 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 01 14:46:43.608672 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Sep 01 14:46:43.608672 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 434/884] 10.222.0.33 () {56 vars in 1117 bytes} [Thu Sep 1 14:46:43 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:46:43.609536 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811cbde1-1400-4360-9529-a8b417ca3ec2 None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:43.611884 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811cbde1-1400-4360-9529-a8b417ca3ec2 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:43.611884 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811cbde1-1400-4360-9529-a8b417ca3ec2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:43.612974 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f90641-1144-4368-bfea-19c5ce919b44 None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:43.628822 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-811cbde1-1400-4360-9529-a8b417ca3ec2 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:46:43.630034 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-811cbde1-1400-4360-9529-a8b417ca3ec2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:46:43.638597 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:43.714917 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 451/885] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 1 14:46:43 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 14:46:43.739817 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f90641-1144-4368-bfea-19c5ce919b44 None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:43.774679 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb30f16-662f-4398-ae32-5846686438c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:43.775564 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb30f16-662f-4398-ae32-5846686438c5 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:43.775997 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb30f16-662f-4398-ae32-5846686438c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:43.776417 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb30f16-662f-4398-ae32-5846686438c5 None admin] PATH_INFO: `/v3/users` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:43.777293 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb30f16-662f-4398-ae32-5846686438c5 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81341) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:46:43.777744 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb30f16-662f-4398-ae32-5846686438c5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:43.779387 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb30f16-662f-4398-ae32-5846686438c5 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:43.799741 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 435/886] 10.222.0.33 () {56 vars in 1149 bytes} [Thu Sep 1 14:46:43 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 14:46:43.807463 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811cbde1-1400-4360-9529-a8b417ca3ec2 None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:43.911767 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811cbde1-1400-4360-9529-a8b417ca3ec2 None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:43.926040 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:43.946850 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0881eebb-2bce-459c-94c5-594d8597d600 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:43.947657 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0881eebb-2bce-459c-94c5-594d8597d600 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:43.948110 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0881eebb-2bce-459c-94c5-594d8597d600 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:43.948439 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0881eebb-2bce-459c-94c5-594d8597d600 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:43.998040 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb30f16-662f-4398-ae32-5846686438c5 None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:44.016882 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0881eebb-2bce-459c-94c5-594d8597d600 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:44.018368 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0881eebb-2bce-459c-94c5-594d8597d600 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:44.097039 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:44.133542 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 452/887] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Sep 1 14:46:43 2022] GET /identity/v3/auth/tokens => generated 1372 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 14:46:44.135852 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0881eebb-2bce-459c-94c5-594d8597d600 None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:44.147895 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb30f16-662f-4398-ae32-5846686438c5 None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:44.155224 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:44.183398 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f51fc04-e3b7-4e7b-92c5-cbe53e8484d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:44.184027 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f51fc04-e3b7-4e7b-92c5-cbe53e8484d4 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:44.184302 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f51fc04-e3b7-4e7b-92c5-cbe53e8484d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:44.184597 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f51fc04-e3b7-4e7b-92c5-cbe53e8484d4 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:44.185182 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f51fc04-e3b7-4e7b-92c5-cbe53e8484d4 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:44.186589 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f51fc04-e3b7-4e7b-92c5-cbe53e8484d4 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:44.196199 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application [None req-9f51fc04-e3b7-4e7b-92c5-cbe53e8484d4 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 14:46:44.196199 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:46:44.196199 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 14:46:44.196199 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 14:46:44.196199 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 14:46:44.196199 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 14:46:44.196199 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 14:46:44.196199 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 14:46:44.196199 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 14:46:44.196199 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Sep 01 14:46:44.196199 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 14:46:44.196199 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Sep 01 14:46:44.196199 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:46:44.196199 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:46:44.196199 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:46:44.196199 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:46:44.196199 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:46:44.197246 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:46:44.197246 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:46:44.197246 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:46:44.197246 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:46:44.197246 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:46:44.197246 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:46:44.197246 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 14:46:44.197246 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 14:46:44.197246 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 14:46:44.197246 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 14:46:44.197246 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:46:44.197246 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:46:44.197246 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:46:44.197246 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:46:44.197246 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:46:44.197246 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:46:44.198119 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:46:44.198119 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application with Lock( Sep 01 14:46:44.198119 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:46:44.198119 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:46:44.198119 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:46:44.198119 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:46:44.198119 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:46:44.198119 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:46:44.198119 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:46:44.198119 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:46:44.198119 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 14:46:44.198119 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 14:46:44.198119 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 14:46:44.198119 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Sep 01 14:46:44.198119 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 436/888] 10.222.0.33 () {56 vars in 1126 bytes} [Thu Sep 1 14:46:43 2022] GET /identity/v3/services/object-store => generated 93 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:46:44.199003 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f48ee2-cd92-424c-b965-2940a4e64b41 None None] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:44.199003 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f48ee2-cd92-424c-b965-2940a4e64b41 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:44.199003 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f48ee2-cd92-424c-b965-2940a4e64b41 None None] PATH_INFO: `/` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:44.200576 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 437/889] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 1 14:46:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:46:44.202995 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f48ee2-cd92-424c-b965-2940a4e64b41 None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:44.255081 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0881eebb-2bce-459c-94c5-594d8597d600 None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:44.291083 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824ee715-d08a-4dca-ae5d-ef3f4278dec9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:44.292012 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824ee715-d08a-4dca-ae5d-ef3f4278dec9 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:44.292012 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824ee715-d08a-4dca-ae5d-ef3f4278dec9 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:44.292214 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824ee715-d08a-4dca-ae5d-ef3f4278dec9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:44.303253 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824ee715-d08a-4dca-ae5d-ef3f4278dec9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:44.304711 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824ee715-d08a-4dca-ae5d-ef3f4278dec9 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:44.312866 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f48ee2-cd92-424c-b965-2940a4e64b41 None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:44.320542 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application [None req-824ee715-d08a-4dca-ae5d-ef3f4278dec9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 14:46:44.320542 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:46:44.320542 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:46:44.320542 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:46:44.320542 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:46:44.320542 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:46:44.320542 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:46:44.320542 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:46:44.320542 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:46:44.320542 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:46:44.320542 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:46:44.320542 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:46:44.320542 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 14:46:44.320542 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 14:46:44.320542 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 14:46:44.320542 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 14:46:44.326374 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:46:44.326374 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:46:44.326374 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:46:44.326374 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:46:44.326374 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:46:44.326374 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:46:44.326374 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:46:44.326374 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application with Lock( Sep 01 14:46:44.326374 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:46:44.326374 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:46:44.326374 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:46:44.326374 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:46:44.326374 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:46:44.326374 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:46:44.326374 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:46:44.326374 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:46:44.326374 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 14:46:44.327550 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 14:46:44.327550 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 14:46:44.327550 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Sep 01 14:46:44.327550 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 453/890] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 1 14:46:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 14:46:44.327550 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7eee09-d2b6-4431-9c91-17b4dfcbf88e None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:44.327550 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7eee09-d2b6-4431-9c91-17b4dfcbf88e None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:44.327550 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7eee09-d2b6-4431-9c91-17b4dfcbf88e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:44.343913 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-2e7eee09-d2b6-4431-9c91-17b4dfcbf88e None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:46:44.345112 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-2e7eee09-d2b6-4431-9c91-17b4dfcbf88e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:46:44.351460 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:44.357856 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be73fd67-0002-466b-9f97-7884600adaea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:44.358506 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be73fd67-0002-466b-9f97-7884600adaea None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:44.358819 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be73fd67-0002-466b-9f97-7884600adaea None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:44.359047 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be73fd67-0002-466b-9f97-7884600adaea None admin] PATH_INFO: `/v3/services` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:44.359670 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be73fd67-0002-466b-9f97-7884600adaea None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81341) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:46:44.359997 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be73fd67-0002-466b-9f97-7884600adaea None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:44.361396 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be73fd67-0002-466b-9f97-7884600adaea None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:44.366612 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 438/891] 10.222.0.33 () {56 vars in 1122 bytes} [Thu Sep 1 14:46:44 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 14:46:44.367211 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be73fd67-0002-466b-9f97-7884600adaea None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:44.414054 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:44.423524 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 454/892] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 1 14:46:44 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 14:46:44.424336 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7eee09-d2b6-4431-9c91-17b4dfcbf88e None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:44.485412 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be73fd67-0002-466b-9f97-7884600adaea None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:44.522792 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf44c1d-c45c-41c0-a498-09987749fa4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:44.523392 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf44c1d-c45c-41c0-a498-09987749fa4d None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:44.523668 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf44c1d-c45c-41c0-a498-09987749fa4d None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:44.523939 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf44c1d-c45c-41c0-a498-09987749fa4d None admin] PATH_INFO: `/v3/domains` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:44.524572 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf44c1d-c45c-41c0-a498-09987749fa4d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81341) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:46:44.524875 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf44c1d-c45c-41c0-a498-09987749fa4d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:44.526266 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf44c1d-c45c-41c0-a498-09987749fa4d None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:44.527386 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7eee09-d2b6-4431-9c91-17b4dfcbf88e None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:44.538344 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 439/893] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Sep 1 14:46:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:46:44.539573 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df835373-d168-4066-878d-2bf9d896ac83 None None] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:44.539852 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df835373-d168-4066-878d-2bf9d896ac83 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:44.540116 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df835373-d168-4066-878d-2bf9d896ac83 None None] PATH_INFO: `/` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:44.541443 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 440/894] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 1 14:46:44 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:46:44.546175 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df835373-d168-4066-878d-2bf9d896ac83 None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:44.570638 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e4e0b6-08ef-4695-ac6d-98f6aa767a01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:44.571566 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e4e0b6-08ef-4695-ac6d-98f6aa767a01 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:44.572042 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e4e0b6-08ef-4695-ac6d-98f6aa767a01 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:44.572487 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e4e0b6-08ef-4695-ac6d-98f6aa767a01 None admin] PATH_INFO: `/v3/services` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:44.573402 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e4e0b6-08ef-4695-ac6d-98f6aa767a01 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:46:44.573982 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e4e0b6-08ef-4695-ac6d-98f6aa767a01 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:44.575551 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e4e0b6-08ef-4695-ac6d-98f6aa767a01 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:44.580543 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 455/895] 10.222.0.33 () {56 vars in 1122 bytes} [Thu Sep 1 14:46:44 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:46:44.581995 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3913f31-2926-4df3-8e08-0a8b1f52c15c None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:44.582295 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3913f31-2926-4df3-8e08-0a8b1f52c15c None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:44.582568 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3913f31-2926-4df3-8e08-0a8b1f52c15c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:44.609618 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-c3913f31-2926-4df3-8e08-0a8b1f52c15c None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:46:44.610795 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-c3913f31-2926-4df3-8e08-0a8b1f52c15c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:46:44.618293 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:44.676851 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df835373-d168-4066-878d-2bf9d896ac83 None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:44.688149 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:44.706940 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 456/896] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 1 14:46:44 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 14:46:44.706940 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3913f31-2926-4df3-8e08-0a8b1f52c15c None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:44.723348 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e286065e-c971-4fc5-9526-5e32992568dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:44.723995 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e286065e-c971-4fc5-9526-5e32992568dc None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:44.724271 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e286065e-c971-4fc5-9526-5e32992568dc None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:44.724537 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e286065e-c971-4fc5-9526-5e32992568dc None admin] PATH_INFO: `/v3/projects/service` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:44.732581 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e286065e-c971-4fc5-9526-5e32992568dc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:44.734080 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e286065e-c971-4fc5-9526-5e32992568dc None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:44.747571 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application [None req-e286065e-c971-4fc5-9526-5e32992568dc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 14:46:44.747571 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:46:44.747571 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:46:44.747571 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:46:44.747571 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:46:44.747571 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:46:44.747571 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:46:44.747571 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:46:44.747571 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:46:44.747571 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:46:44.747571 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:46:44.747571 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:46:44.747571 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 14:46:44.747571 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 14:46:44.747571 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 14:46:44.748791 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 14:46:44.748791 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:46:44.748791 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:46:44.748791 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:46:44.748791 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:46:44.748791 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:46:44.748791 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:46:44.748791 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:46:44.748791 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application with Lock( Sep 01 14:46:44.748791 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:46:44.748791 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:46:44.748791 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:46:44.748791 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:46:44.748791 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:46:44.748791 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:46:44.748791 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:46:44.748791 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:46:44.749938 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 14:46:44.749938 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 14:46:44.749938 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 14:46:44.749938 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 14:46:44.749938 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 14:46:44.749938 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 14:46:44.749938 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 14:46:44.749938 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Sep 01 14:46:44.749938 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 441/897] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Sep 1 14:46:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:46:44.754645 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e286065e-c971-4fc5-9526-5e32992568dc None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:44.832682 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3913f31-2926-4df3-8e08-0a8b1f52c15c None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:44.869535 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ae49db-2ce2-488e-90ff-a0e1a30e2e54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:44.870550 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ae49db-2ce2-488e-90ff-a0e1a30e2e54 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:44.871087 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ae49db-2ce2-488e-90ff-a0e1a30e2e54 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:44.871580 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ae49db-2ce2-488e-90ff-a0e1a30e2e54 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:44.872368 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ae49db-2ce2-488e-90ff-a0e1a30e2e54 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:44.873934 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ae49db-2ce2-488e-90ff-a0e1a30e2e54 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:44.883243 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application [None req-d4ae49db-2ce2-488e-90ff-a0e1a30e2e54 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 01 14:46:44.883243 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:46:44.883243 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 14:46:44.883243 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 14:46:44.883243 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 14:46:44.883243 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 14:46:44.883243 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 14:46:44.883243 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 14:46:44.883243 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 01 14:46:44.883243 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Sep 01 14:46:44.883243 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 14:46:44.883243 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Sep 01 14:46:44.883243 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:46:44.883243 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:46:44.883243 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:46:44.883243 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:46:44.883243 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:46:44.884271 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:46:44.884271 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:46:44.884271 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:46:44.884271 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:46:44.884271 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:46:44.884271 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:46:44.884271 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 14:46:44.884271 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 14:46:44.884271 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 14:46:44.884271 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 14:46:44.884271 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:46:44.884271 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:46:44.884271 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:46:44.884271 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:46:44.884271 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:46:44.884271 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:46:44.885170 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:46:44.885170 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application with Lock( Sep 01 14:46:44.885170 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:46:44.885170 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:46:44.885170 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:46:44.885170 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:46:44.885170 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:46:44.885170 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:46:44.885170 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:46:44.885170 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:46:44.885170 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 14:46:44.885170 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 14:46:44.885170 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 01 14:46:44.885170 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Sep 01 14:46:44.887209 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e286065e-c971-4fc5-9526-5e32992568dc None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:44.888188 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 457/898] 10.222.0.33 () {56 vars in 1114 bytes} [Thu Sep 1 14:46:44 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:46:44.893989 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ae49db-2ce2-488e-90ff-a0e1a30e2e54 None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:44.904856 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:44.926507 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40192e85-2738-4f18-848b-6ac8f75b64e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:44.927121 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40192e85-2738-4f18-848b-6ac8f75b64e6 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:44.927400 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40192e85-2738-4f18-848b-6ac8f75b64e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:44.927671 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40192e85-2738-4f18-848b-6ac8f75b64e6 None admin] PATH_INFO: `/v3/projects` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:44.928467 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40192e85-2738-4f18-848b-6ac8f75b64e6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81341) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:46:44.928769 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40192e85-2738-4f18-848b-6ac8f75b64e6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:44.930355 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40192e85-2738-4f18-848b-6ac8f75b64e6 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:44.943650 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 442/899] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Sep 1 14:46:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 14:46:44.949451 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40192e85-2738-4f18-848b-6ac8f75b64e6 None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:44.955394 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:45.004329 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ae49db-2ce2-488e-90ff-a0e1a30e2e54 None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:45.041173 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20866da8-4dbb-4aa0-a7f8-d92b783ccc1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:45.041777 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20866da8-4dbb-4aa0-a7f8-d92b783ccc1a None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:45.042053 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20866da8-4dbb-4aa0-a7f8-d92b783ccc1a None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:45.042345 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20866da8-4dbb-4aa0-a7f8-d92b783ccc1a None admin] PATH_INFO: `/v3/services` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:45.042971 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20866da8-4dbb-4aa0-a7f8-d92b783ccc1a None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:46:45.043292 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20866da8-4dbb-4aa0-a7f8-d92b783ccc1a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:45.044656 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20866da8-4dbb-4aa0-a7f8-d92b783ccc1a None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:45.049626 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 458/900] 10.222.0.33 () {56 vars in 1114 bytes} [Thu Sep 1 14:46:44 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 14:46:45.050439 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20866da8-4dbb-4aa0-a7f8-d92b783ccc1a None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:45.080000 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40192e85-2738-4f18-848b-6ac8f75b64e6 None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:45.116487 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bbf767-4338-4c47-b81a-a769a286fe75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:45.117093 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bbf767-4338-4c47-b81a-a769a286fe75 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:45.117402 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bbf767-4338-4c47-b81a-a769a286fe75 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:45.117706 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bbf767-4338-4c47-b81a-a769a286fe75 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:45.118386 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bbf767-4338-4c47-b81a-a769a286fe75 None admin] RBAC: Adding query filter params (role.id=050916e5d27c476c9eed7850b136dd98, scope.project.id=e8c85b4bd17e4e89b56bf0f831b474d5, user.id=24b6f275ffb04ffeae3c2097bfd040f5) {{(pid=81341) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:46:45.118692 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bbf767-4338-4c47-b81a-a769a286fe75 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:45.120175 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bbf767-4338-4c47-b81a-a769a286fe75 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:45.125694 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 443/901] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Sep 1 14:46:44 2022] GET /identity/v3/role_assignments?user.id=24b6f275ffb04ffeae3c2097bfd040f5&scope.project.id=e8c85b4bd17e4e89b56bf0f831b474d5&role.id=050916e5d27c476c9eed7850b136dd98 => generated 259 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:46:45.126220 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bbf767-4338-4c47-b81a-a769a286fe75 None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:45.176190 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20866da8-4dbb-4aa0-a7f8-d92b783ccc1a None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:45.196088 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:45.201477 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:45.226687 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201c018d-a806-470f-84d2-d40b18c534d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:45.227345 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201c018d-a806-470f-84d2-d40b18c534d3 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:45.227653 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201c018d-a806-470f-84d2-d40b18c534d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:45.228244 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201c018d-a806-470f-84d2-d40b18c534d3 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:45.228863 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201c018d-a806-470f-84d2-d40b18c534d3 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:45.231131 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201c018d-a806-470f-84d2-d40b18c534d3 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:45.243405 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application [None req-201c018d-a806-470f-84d2-d40b18c534d3 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 01 14:46:45.243405 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:46:45.243405 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 14:46:45.243405 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 14:46:45.243405 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 14:46:45.243405 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 14:46:45.243405 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 14:46:45.243405 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 14:46:45.243405 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 01 14:46:45.243405 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Sep 01 14:46:45.243405 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 14:46:45.243405 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Sep 01 14:46:45.243405 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:46:45.243405 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:46:45.243405 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:46:45.243405 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:46:45.249453 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:46:45.249453 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:46:45.249453 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:46:45.249453 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:46:45.249453 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:46:45.249453 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:46:45.249453 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:46:45.249453 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 14:46:45.249453 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 14:46:45.249453 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 14:46:45.249453 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 14:46:45.249453 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:46:45.249453 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:46:45.249453 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:46:45.249453 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:46:45.249453 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:46:45.249453 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:46:45.250579 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:46:45.250579 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application with Lock( Sep 01 14:46:45.250579 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:46:45.250579 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:46:45.250579 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:46:45.250579 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:46:45.250579 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:46:45.250579 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:46:45.250579 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:46:45.250579 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:46:45.250579 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 14:46:45.250579 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 14:46:45.250579 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 01 14:46:45.250579 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Sep 01 14:46:45.250579 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 459/902] 10.222.0.33 () {56 vars in 1132 bytes} [Thu Sep 1 14:46:45 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:46:45.252830 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201c018d-a806-470f-84d2-d40b18c534d3 None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:45.277641 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bbf767-4338-4c47-b81a-a769a286fe75 None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:45.314498 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38141546-b45a-4f9c-b91c-250e4f689e98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:45.315170 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38141546-b45a-4f9c-b91c-250e4f689e98 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:45.315380 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38141546-b45a-4f9c-b91c-250e4f689e98 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:45.315653 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38141546-b45a-4f9c-b91c-250e4f689e98 None admin] PATH_INFO: `/v3/services` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:45.316802 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38141546-b45a-4f9c-b91c-250e4f689e98 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81341) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:46:45.317109 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38141546-b45a-4f9c-b91c-250e4f689e98 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:45.318533 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38141546-b45a-4f9c-b91c-250e4f689e98 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:45.324704 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 444/903] 10.222.0.33 () {56 vars in 1114 bytes} [Thu Sep 1 14:46:45 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:46:45.330107 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38141546-b45a-4f9c-b91c-250e4f689e98 None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:45.370166 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201c018d-a806-470f-84d2-d40b18c534d3 None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:45.410092 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf6a646-6eef-4d04-ba79-c5d068f59b26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:45.410798 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf6a646-6eef-4d04-ba79-c5d068f59b26 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:45.411231 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf6a646-6eef-4d04-ba79-c5d068f59b26 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:45.411616 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf6a646-6eef-4d04-ba79-c5d068f59b26 None admin] PATH_INFO: `/v3/services` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:45.412466 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf6a646-6eef-4d04-ba79-c5d068f59b26 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:46:45.412746 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf6a646-6eef-4d04-ba79-c5d068f59b26 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:45.414226 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf6a646-6eef-4d04-ba79-c5d068f59b26 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:45.423898 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 460/904] 10.222.0.33 () {56 vars in 1126 bytes} [Thu Sep 1 14:46:45 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:46:45.426223 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf6a646-6eef-4d04-ba79-c5d068f59b26 None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:45.462038 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38141546-b45a-4f9c-b91c-250e4f689e98 None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:45.465326 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:45.491692 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:45.510471 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5619c1-1f68-499d-b288-47f2329f7900 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:45.511817 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5619c1-1f68-499d-b288-47f2329f7900 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:45.512506 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5619c1-1f68-499d-b288-47f2329f7900 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:45.513044 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5619c1-1f68-499d-b288-47f2329f7900 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:45.514347 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5619c1-1f68-499d-b288-47f2329f7900 None admin] RBAC: Adding query filter params (interface=public, service_id=519a4e6f899e4d209091cf9594e06969, region_id=RegionOne) {{(pid=81341) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:46:45.515245 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5619c1-1f68-499d-b288-47f2329f7900 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:45.518213 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5619c1-1f68-499d-b288-47f2329f7900 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:45.527746 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 445/905] 10.222.0.33 () {56 vars in 1251 bytes} [Thu Sep 1 14:46:45 2022] GET /identity/v3/endpoints?service_id=519a4e6f899e4d209091cf9594e06969&interface=public®ion_id=RegionOne => generated 194 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 14:46:45.533835 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5619c1-1f68-499d-b288-47f2329f7900 None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:45.558862 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf6a646-6eef-4d04-ba79-c5d068f59b26 None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:45.597954 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f638afe0-c210-474e-90dd-474bdcf38ba1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:45.598785 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f638afe0-c210-474e-90dd-474bdcf38ba1 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:45.599185 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f638afe0-c210-474e-90dd-474bdcf38ba1 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:45.599658 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f638afe0-c210-474e-90dd-474bdcf38ba1 None admin] PATH_INFO: `/v3/services` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:45.600687 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f638afe0-c210-474e-90dd-474bdcf38ba1 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:46:45.600872 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f638afe0-c210-474e-90dd-474bdcf38ba1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:45.602366 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f638afe0-c210-474e-90dd-474bdcf38ba1 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:45.608421 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 461/906] 10.222.0.33 () {56 vars in 1126 bytes} [Thu Sep 1 14:46:45 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:46:45.613903 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f638afe0-c210-474e-90dd-474bdcf38ba1 None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:45.652157 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5619c1-1f68-499d-b288-47f2329f7900 None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:45.697813 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ba72d2-54d6-46f7-801f-0e76576f752e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:45.698452 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ba72d2-54d6-46f7-801f-0e76576f752e None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:45.698784 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ba72d2-54d6-46f7-801f-0e76576f752e None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:45.699112 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ba72d2-54d6-46f7-801f-0e76576f752e None admin] PATH_INFO: `/v3/services` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:45.700173 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ba72d2-54d6-46f7-801f-0e76576f752e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:45.701987 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ba72d2-54d6-46f7-801f-0e76576f752e None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:45.710699 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 446/907] 10.222.0.33 () {56 vars in 1087 bytes} [Thu Sep 1 14:46:45 2022] GET /identity/v3/services => generated 1268 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 14:46:45.733608 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f638afe0-c210-474e-90dd-474bdcf38ba1 None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:45.746805 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:45.775149 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21eaa91-76de-4695-a075-6d5894b0ce48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:45.775770 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21eaa91-76de-4695-a075-6d5894b0ce48 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:45.776043 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21eaa91-76de-4695-a075-6d5894b0ce48 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:45.776307 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21eaa91-76de-4695-a075-6d5894b0ce48 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:45.777026 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21eaa91-76de-4695-a075-6d5894b0ce48 None admin] RBAC: Adding query filter params (interface=public, service_id=3360a64158be434a97b9e167fba801e4, region_id=RegionOne) {{(pid=81342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:46:45.777329 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21eaa91-76de-4695-a075-6d5894b0ce48 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:45.778901 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21eaa91-76de-4695-a075-6d5894b0ce48 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:45.790203 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 462/908] 10.222.0.33 () {56 vars in 1251 bytes} [Thu Sep 1 14:46:45 2022] GET /identity/v3/endpoints?service_id=3360a64158be434a97b9e167fba801e4&interface=public®ion_id=RegionOne => generated 194 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:46:45.792397 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ba72d2-54d6-46f7-801f-0e76576f752e None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:45.829491 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:45.902766 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ba72d2-54d6-46f7-801f-0e76576f752e None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:45.937729 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4551e1e0-1ad6-4699-9ece-dcc9d8e96ed5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:45.938339 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4551e1e0-1ad6-4699-9ece-dcc9d8e96ed5 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:45.938618 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4551e1e0-1ad6-4699-9ece-dcc9d8e96ed5 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:45.938886 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4551e1e0-1ad6-4699-9ece-dcc9d8e96ed5 None admin] PATH_INFO: `/v3/services` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:45.939535 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4551e1e0-1ad6-4699-9ece-dcc9d8e96ed5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:45.940909 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4551e1e0-1ad6-4699-9ece-dcc9d8e96ed5 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:45.950709 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 447/909] 10.222.0.33 () {56 vars in 1087 bytes} [Thu Sep 1 14:46:45 2022] GET /identity/v3/services => generated 1268 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:46:46.572057 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7eb5b89-0023-4f4c-a1a4-75bad2eef9eb None None] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:46.572461 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7eb5b89-0023-4f4c-a1a4-75bad2eef9eb None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:46.572587 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7eb5b89-0023-4f4c-a1a4-75bad2eef9eb None None] PATH_INFO: `/` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:46.574295 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 463/910] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 1 14:46:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:46:46.579697 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47424f21-973d-45a6-8eb6-80ae0226273a None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:46.580140 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47424f21-973d-45a6-8eb6-80ae0226273a None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:46.580308 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47424f21-973d-45a6-8eb6-80ae0226273a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:46.600191 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-47424f21-973d-45a6-8eb6-80ae0226273a None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:46:46.601383 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-47424f21-973d-45a6-8eb6-80ae0226273a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:46:46.674343 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 448/911] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 1 14:46:46 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 14:46:46.937601 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7eb5b89-0023-4f4c-a1a4-75bad2eef9eb None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:47.046962 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7eb5b89-0023-4f4c-a1a4-75bad2eef9eb None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:47.083725 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203380f3-b62d-4782-a647-a8ee288b2abc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:47.084339 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203380f3-b62d-4782-a647-a8ee288b2abc None admin] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:47.084653 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203380f3-b62d-4782-a647-a8ee288b2abc None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:47.084917 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203380f3-b62d-4782-a647-a8ee288b2abc None admin] PATH_INFO: `/v3/services` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:47.085622 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203380f3-b62d-4782-a647-a8ee288b2abc None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:47.087030 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203380f3-b62d-4782-a647-a8ee288b2abc None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:47.104750 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f09fd4-135b-4be8-8428-a4bb92883f2f None None] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:47.105061 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f09fd4-135b-4be8-8428-a4bb92883f2f None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:47.105302 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f09fd4-135b-4be8-8428-a4bb92883f2f None None] PATH_INFO: `/` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:47.106925 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 449/913] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 1 14:46:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:46:47.108003 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 464/913] 10.222.0.33 () {60 vars in 1141 bytes} [Thu Sep 1 14:46:46 2022] POST /identity/v3/services => generated 241 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:46:47.112225 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c821496-4243-4851-bb12-5b944b834c9a None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:47.112504 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c821496-4243-4851-bb12-5b944b834c9a None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:47.112791 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c821496-4243-4851-bb12-5b944b834c9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:47.132883 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-2c821496-4243-4851-bb12-5b944b834c9a None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:46:47.134125 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-2c821496-4243-4851-bb12-5b944b834c9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:46:47.158153 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:47.215791 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 450/914] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 1 14:46:47 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 14:46:47.483702 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203380f3-b62d-4782-a647-a8ee288b2abc None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:47.521435 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:47.593287 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203380f3-b62d-4782-a647-a8ee288b2abc None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:47.629792 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f770821-c5ca-42a9-bb96-4c74221a6b45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:47.630449 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f770821-c5ca-42a9-bb96-4c74221a6b45 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:47.630729 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f770821-c5ca-42a9-bb96-4c74221a6b45 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:47.631001 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f770821-c5ca-42a9-bb96-4c74221a6b45 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:47.641566 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f770821-c5ca-42a9-bb96-4c74221a6b45 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 14:46:47.646433 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application [None req-9f770821-c5ca-42a9-bb96-4c74221a6b45 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 01 14:46:47.646433 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:46:47.646433 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:46:47.646433 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:46:47.646433 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:46:47.646433 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:46:47.646433 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:46:47.646433 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:46:47.646433 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:46:47.646433 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:46:47.646433 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:46:47.646433 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:46:47.646433 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 14:46:47.646433 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 14:46:47.646433 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 14:46:47.646433 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 14:46:47.648541 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 14:46:47.648541 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 14:46:47.648541 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 14:46:47.648541 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 14:46:47.648541 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:46:47.648541 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:46:47.648541 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:46:47.648541 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:46:47.648541 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:46:47.648541 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:46:47.648541 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:46:47.648541 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application with Lock( Sep 01 14:46:47.648541 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:46:47.648541 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:46:47.648541 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:46:47.648541 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:46:47.650634 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:46:47.650634 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:46:47.650634 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:46:47.650634 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:46:47.650634 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 14:46:47.650634 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 14:46:47.650634 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 14:46:47.650634 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 14:46:47.650634 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 14:46:47.650634 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 14:46:47.650634 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 01 14:46:47.650634 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Sep 01 14:46:47.650634 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 465/915] 10.222.0.33 () {56 vars in 1120 bytes} [Thu Sep 1 14:46:47 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:46:47.654267 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c821496-4243-4851-bb12-5b944b834c9a None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:47.695631 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f46e86-61f5-408f-ae17-d1356b493829 None None] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:47.696059 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f46e86-61f5-408f-ae17-d1356b493829 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:47.696344 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f46e86-61f5-408f-ae17-d1356b493829 None None] PATH_INFO: `/` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:47.699122 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 466/916] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 1 14:46:47 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:46:47.704171 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbee3ea9-820d-4758-8798-1d9b78a105ae None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:47.704455 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbee3ea9-820d-4758-8798-1d9b78a105ae None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:47.704731 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbee3ea9-820d-4758-8798-1d9b78a105ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:47.724154 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-dbee3ea9-820d-4758-8798-1d9b78a105ae None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:46:47.725452 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-dbee3ea9-820d-4758-8798-1d9b78a105ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:46:47.771360 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c821496-4243-4851-bb12-5b944b834c9a None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:47.809251 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4dabd5-2742-4757-8232-9808b6f5129f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:47.809859 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4dabd5-2742-4757-8232-9808b6f5129f None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:47.810140 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4dabd5-2742-4757-8232-9808b6f5129f None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:47.810429 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4dabd5-2742-4757-8232-9808b6f5129f None admin] PATH_INFO: `/v3/roles` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:47.811092 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4dabd5-2742-4757-8232-9808b6f5129f None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81341) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:46:47.811387 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4dabd5-2742-4757-8232-9808b6f5129f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:47.812760 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4dabd5-2742-4757-8232-9808b6f5129f None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:47.818827 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 467/917] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 1 14:46:47 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 14:46:47.824850 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 451/918] 10.222.0.33 () {56 vars in 1115 bytes} [Thu Sep 1 14:46:47 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:46:47.831234 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbee3ea9-820d-4758-8798-1d9b78a105ae None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:47.857173 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:47.901686 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819278da-2b2b-4637-990e-c2e5854beb0b None None] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:47.901686 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819278da-2b2b-4637-990e-c2e5854beb0b None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:47.901686 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819278da-2b2b-4637-990e-c2e5854beb0b None None] PATH_INFO: `/` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:47.902558 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 452/919] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 1 14:46:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:46:47.907447 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9521b9ad-1cf2-431e-80cc-daac191caa15 None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:47.907963 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9521b9ad-1cf2-431e-80cc-daac191caa15 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:47.908455 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9521b9ad-1cf2-431e-80cc-daac191caa15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:47.927410 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-9521b9ad-1cf2-431e-80cc-daac191caa15 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:46:47.928892 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-9521b9ad-1cf2-431e-80cc-daac191caa15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:46:47.938046 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbee3ea9-820d-4758-8798-1d9b78a105ae None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:47.939188 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:47.977746 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a0a51f-d62e-4284-89f0-6beff6eca373 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:47.978411 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a0a51f-d62e-4284-89f0-6beff6eca373 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:47.978697 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a0a51f-d62e-4284-89f0-6beff6eca373 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:47.979340 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a0a51f-d62e-4284-89f0-6beff6eca373 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:47.991102 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a0a51f-d62e-4284-89f0-6beff6eca373 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:47.993278 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a0a51f-d62e-4284-89f0-6beff6eca373 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:48.007586 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 453/920] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 1 14:46:47 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 14:46:48.010741 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application [None req-22a0a51f-d62e-4284-89f0-6beff6eca373 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 14:46:48.010741 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:46:48.010741 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:46:48.010741 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:46:48.010741 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:46:48.010741 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:46:48.010741 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:46:48.010741 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:46:48.010741 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:46:48.010741 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:46:48.010741 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:46:48.010741 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:46:48.010741 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 14:46:48.010741 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 14:46:48.010741 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 14:46:48.010741 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 14:46:48.011692 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:46:48.011692 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:46:48.011692 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:46:48.011692 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:46:48.011692 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:46:48.011692 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:46:48.011692 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:46:48.011692 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application with Lock( Sep 01 14:46:48.011692 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:46:48.011692 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:46:48.011692 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:46:48.011692 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:46:48.011692 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:46:48.011692 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:46:48.011692 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:46:48.011692 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:46:48.011692 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 14:46:48.012616 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 14:46:48.012616 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 14:46:48.012616 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Sep 01 14:46:48.012616 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 468/921] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 1 14:46:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:46:48.017662 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9521b9ad-1cf2-431e-80cc-daac191caa15 None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:48.111208 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a0a51f-d62e-4284-89f0-6beff6eca373 None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:48.119238 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9521b9ad-1cf2-431e-80cc-daac191caa15 None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:48.154741 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb4f99c-0c96-4be0-a977-2382068393b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:48.155351 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb4f99c-0c96-4be0-a977-2382068393b8 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:48.155633 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb4f99c-0c96-4be0-a977-2382068393b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:48.155905 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb4f99c-0c96-4be0-a977-2382068393b8 None admin] PATH_INFO: `/v3/domains` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:48.156554 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb4f99c-0c96-4be0-a977-2382068393b8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81341) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:46:48.156855 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb4f99c-0c96-4be0-a977-2382068393b8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:48.158261 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb4f99c-0c96-4be0-a977-2382068393b8 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:48.170705 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 454/922] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Sep 1 14:46:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:46:48.176195 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb4f99c-0c96-4be0-a977-2382068393b8 None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:48.180213 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:48.197463 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:48.223494 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a0a51f-d62e-4284-89f0-6beff6eca373 None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:48.260400 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748fc1cf-9d8e-447e-b389-67e8c8a91132 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:48.261100 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748fc1cf-9d8e-447e-b389-67e8c8a91132 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:48.261378 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748fc1cf-9d8e-447e-b389-67e8c8a91132 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:48.261643 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748fc1cf-9d8e-447e-b389-67e8c8a91132 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:48.262266 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748fc1cf-9d8e-447e-b389-67e8c8a91132 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:48.263868 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748fc1cf-9d8e-447e-b389-67e8c8a91132 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:48.274123 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application [None req-748fc1cf-9d8e-447e-b389-67e8c8a91132 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 01 14:46:48.274123 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:46:48.274123 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 14:46:48.274123 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 14:46:48.274123 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 14:46:48.274123 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 14:46:48.274123 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 14:46:48.274123 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 14:46:48.274123 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 01 14:46:48.274123 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Sep 01 14:46:48.274123 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 14:46:48.274123 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Sep 01 14:46:48.274123 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:46:48.274123 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:46:48.274123 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:46:48.274123 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:46:48.274123 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:46:48.275156 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:46:48.275156 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:46:48.275156 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:46:48.275156 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:46:48.275156 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:46:48.275156 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:46:48.275156 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 14:46:48.275156 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 14:46:48.275156 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 14:46:48.275156 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 14:46:48.275156 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:46:48.275156 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:46:48.275156 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:46:48.275156 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:46:48.275156 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:46:48.275156 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:46:48.276031 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:46:48.276031 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application with Lock( Sep 01 14:46:48.276031 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:46:48.276031 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:46:48.276031 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:46:48.276031 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:46:48.276031 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:46:48.276031 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:46:48.276031 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:46:48.276031 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:46:48.276031 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 14:46:48.276031 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 14:46:48.276031 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 01 14:46:48.276031 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Sep 01 14:46:48.280214 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 469/923] 10.222.0.33 () {56 vars in 1114 bytes} [Thu Sep 1 14:46:48 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 14:46:48.281186 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748fc1cf-9d8e-447e-b389-67e8c8a91132 None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:48.282669 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb4f99c-0c96-4be0-a977-2382068393b8 None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:48.318384 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b655b8-ddf3-4dc3-8f9b-4e2776fac529 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:48.319015 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b655b8-ddf3-4dc3-8f9b-4e2776fac529 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:48.319295 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b655b8-ddf3-4dc3-8f9b-4e2776fac529 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:48.319568 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b655b8-ddf3-4dc3-8f9b-4e2776fac529 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:48.327645 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b655b8-ddf3-4dc3-8f9b-4e2776fac529 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:48.329059 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b655b8-ddf3-4dc3-8f9b-4e2776fac529 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:48.340769 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application [None req-43b655b8-ddf3-4dc3-8f9b-4e2776fac529 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 01 14:46:48.340769 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:46:48.340769 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:46:48.340769 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:46:48.340769 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:46:48.340769 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:46:48.340769 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:46:48.340769 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:46:48.340769 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:46:48.340769 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:46:48.340769 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:46:48.340769 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:46:48.340769 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 14:46:48.340769 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 14:46:48.340769 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 14:46:48.340769 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 14:46:48.341756 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:46:48.341756 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:46:48.341756 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 14:46:48.341756 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 14:46:48.341756 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 14:46:48.341756 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 14:46:48.341756 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:46:48.341756 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:46:48.341756 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:46:48.341756 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:46:48.341756 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:46:48.341756 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application with Lock( Sep 01 14:46:48.341756 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:46:48.341756 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:46:48.341756 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:46:48.341756 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:46:48.341756 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:46:48.341756 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:46:48.343263 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:46:48.343263 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:46:48.343263 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 14:46:48.343263 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 14:46:48.343263 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 14:46:48.343263 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 14:46:48.343263 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 14:46:48.343263 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 14:46:48.343263 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 01 14:46:48.343263 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Sep 01 14:46:48.343263 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 455/924] 10.222.0.33 () {56 vars in 1117 bytes} [Thu Sep 1 14:46:48 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 14:46:48.343263 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b655b8-ddf3-4dc3-8f9b-4e2776fac529 None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:48.388274 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748fc1cf-9d8e-447e-b389-67e8c8a91132 None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:48.409658 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:48.424933 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13517010-4dde-4345-8f4a-431131b69d6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:48.425552 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13517010-4dde-4345-8f4a-431131b69d6c None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:48.425831 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13517010-4dde-4345-8f4a-431131b69d6c None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:48.426346 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13517010-4dde-4345-8f4a-431131b69d6c None admin] PATH_INFO: `/v3/services` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:48.426752 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13517010-4dde-4345-8f4a-431131b69d6c None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:46:48.427046 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13517010-4dde-4345-8f4a-431131b69d6c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:48.428400 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13517010-4dde-4345-8f4a-431131b69d6c None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:48.437553 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 470/925] 10.222.0.33 () {56 vars in 1114 bytes} [Thu Sep 1 14:46:48 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:46:48.437553 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13517010-4dde-4345-8f4a-431131b69d6c None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:48.442076 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:48.455299 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b655b8-ddf3-4dc3-8f9b-4e2776fac529 None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:48.490475 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c820db-e558-478b-97bb-42ea9bfe24b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:48.491103 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c820db-e558-478b-97bb-42ea9bfe24b5 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:48.491389 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c820db-e558-478b-97bb-42ea9bfe24b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:48.491661 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c820db-e558-478b-97bb-42ea9bfe24b5 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:48.492239 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c820db-e558-478b-97bb-42ea9bfe24b5 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:48.493623 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c820db-e558-478b-97bb-42ea9bfe24b5 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:48.504808 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application [None req-02c820db-e558-478b-97bb-42ea9bfe24b5 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 01 14:46:48.504808 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:46:48.504808 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 14:46:48.504808 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 14:46:48.504808 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 14:46:48.504808 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 14:46:48.504808 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 14:46:48.504808 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 14:46:48.504808 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 01 14:46:48.504808 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Sep 01 14:46:48.504808 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 14:46:48.504808 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Sep 01 14:46:48.504808 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:46:48.504808 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:46:48.504808 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:46:48.504808 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:46:48.505769 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:46:48.505769 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:46:48.505769 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:46:48.505769 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:46:48.505769 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:46:48.505769 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:46:48.505769 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:46:48.505769 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 14:46:48.505769 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 14:46:48.505769 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 14:46:48.505769 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 14:46:48.505769 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:46:48.505769 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:46:48.505769 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:46:48.505769 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:46:48.505769 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:46:48.505769 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:46:48.511853 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:46:48.511853 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application with Lock( Sep 01 14:46:48.511853 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:46:48.511853 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:46:48.511853 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:46:48.511853 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:46:48.511853 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:46:48.511853 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:46:48.511853 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:46:48.511853 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:46:48.511853 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 14:46:48.511853 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 14:46:48.511853 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 01 14:46:48.511853 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Sep 01 14:46:48.511853 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 456/926] 10.222.0.33 () {56 vars in 1132 bytes} [Thu Sep 1 14:46:48 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:46:48.512740 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c820db-e558-478b-97bb-42ea9bfe24b5 None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:48.544548 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13517010-4dde-4345-8f4a-431131b69d6c None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:48.581008 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdca577-25b6-443e-8b3d-425f1d25ad10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:48.581611 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdca577-25b6-443e-8b3d-425f1d25ad10 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:48.582546 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdca577-25b6-443e-8b3d-425f1d25ad10 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:48.582825 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdca577-25b6-443e-8b3d-425f1d25ad10 None admin] PATH_INFO: `/v3/users` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:48.583551 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdca577-25b6-443e-8b3d-425f1d25ad10 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:46:48.583865 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdca577-25b6-443e-8b3d-425f1d25ad10 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:48.585285 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdca577-25b6-443e-8b3d-425f1d25ad10 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:48.602810 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 471/927] 10.222.0.33 () {56 vars in 1149 bytes} [Thu Sep 1 14:46:48 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:46:48.603632 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdca577-25b6-443e-8b3d-425f1d25ad10 None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:48.627464 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c820db-e558-478b-97bb-42ea9bfe24b5 None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:48.657386 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:48.668298 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49145492-69c0-4139-bdd4-92584145e4d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:48.668906 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49145492-69c0-4139-bdd4-92584145e4d3 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:48.669191 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49145492-69c0-4139-bdd4-92584145e4d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:48.669457 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49145492-69c0-4139-bdd4-92584145e4d3 None admin] PATH_INFO: `/v3/services` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:48.669682 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:48.670089 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49145492-69c0-4139-bdd4-92584145e4d3 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81341) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:46:48.670418 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49145492-69c0-4139-bdd4-92584145e4d3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:48.671807 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49145492-69c0-4139-bdd4-92584145e4d3 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:48.681171 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 457/928] 10.222.0.33 () {56 vars in 1114 bytes} [Thu Sep 1 14:46:48 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:46:48.681171 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49145492-69c0-4139-bdd4-92584145e4d3 None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:48.718979 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdca577-25b6-443e-8b3d-425f1d25ad10 None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:48.756818 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0c9751-390a-4d8b-ad56-05c9ef1e33d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:48.757420 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0c9751-390a-4d8b-ad56-05c9ef1e33d9 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:48.757695 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0c9751-390a-4d8b-ad56-05c9ef1e33d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:48.758345 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0c9751-390a-4d8b-ad56-05c9ef1e33d9 None admin] PATH_INFO: `/v3/services` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:48.758978 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0c9751-390a-4d8b-ad56-05c9ef1e33d9 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:46:48.759573 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0c9751-390a-4d8b-ad56-05c9ef1e33d9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:48.760944 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0c9751-390a-4d8b-ad56-05c9ef1e33d9 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:48.771004 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 472/929] 10.222.0.33 () {56 vars in 1126 bytes} [Thu Sep 1 14:46:48 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:46:48.771004 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0c9751-390a-4d8b-ad56-05c9ef1e33d9 None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:48.790957 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49145492-69c0-4139-bdd4-92584145e4d3 None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:48.829472 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90380ef3-a8ca-4da2-b241-9f4ccc081648 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:48.830098 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90380ef3-a8ca-4da2-b241-9f4ccc081648 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:48.830415 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90380ef3-a8ca-4da2-b241-9f4ccc081648 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:48.830688 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90380ef3-a8ca-4da2-b241-9f4ccc081648 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:48.841059 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90380ef3-a8ca-4da2-b241-9f4ccc081648 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:48.842544 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90380ef3-a8ca-4da2-b241-9f4ccc081648 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:48.856533 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application [None req-90380ef3-a8ca-4da2-b241-9f4ccc081648 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 14:46:48.856533 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:46:48.856533 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:46:48.856533 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:46:48.856533 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:46:48.856533 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:46:48.856533 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:46:48.856533 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:46:48.856533 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:46:48.856533 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:46:48.856533 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:46:48.856533 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:46:48.856533 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 14:46:48.856533 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 14:46:48.856533 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 14:46:48.856533 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 14:46:48.857518 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:46:48.857518 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:46:48.857518 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:46:48.857518 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:46:48.857518 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:46:48.857518 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:46:48.857518 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:46:48.857518 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application with Lock( Sep 01 14:46:48.857518 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:46:48.857518 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:46:48.857518 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:46:48.857518 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:46:48.857518 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:46:48.857518 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:46:48.857518 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:46:48.857518 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:46:48.857518 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 14:46:48.858475 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 14:46:48.858475 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 14:46:48.858475 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Sep 01 14:46:48.858475 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 458/930] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 1 14:46:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:46:48.858800 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90380ef3-a8ca-4da2-b241-9f4ccc081648 None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:48.878371 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0c9751-390a-4d8b-ad56-05c9ef1e33d9 None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:48.906836 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:48.907845 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:48.917660 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3301e0-f254-4908-a60e-8105049c4f55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:48.918283 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3301e0-f254-4908-a60e-8105049c4f55 None admin] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:48.918616 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3301e0-f254-4908-a60e-8105049c4f55 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:48.918886 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3301e0-f254-4908-a60e-8105049c4f55 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:48.919571 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3301e0-f254-4908-a60e-8105049c4f55 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:48.920946 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3301e0-f254-4908-a60e-8105049c4f55 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:48.942479 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 473/931] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Sep 1 14:46:48 2022] POST /identity/v3/endpoints => generated 346 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:46:48.943980 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3301e0-f254-4908-a60e-8105049c4f55 None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:48.977768 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90380ef3-a8ca-4da2-b241-9f4ccc081648 None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:49.028136 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb638c9-5d79-407a-bc4d-14e936152f92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:49.029346 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb638c9-5d79-407a-bc4d-14e936152f92 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:49.029903 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb638c9-5d79-407a-bc4d-14e936152f92 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:49.030454 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb638c9-5d79-407a-bc4d-14e936152f92 None admin] PATH_INFO: `/v3/services` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:49.031428 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb638c9-5d79-407a-bc4d-14e936152f92 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81341) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:46:49.031957 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb638c9-5d79-407a-bc4d-14e936152f92 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:49.033575 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb638c9-5d79-407a-bc4d-14e936152f92 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:49.044918 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 459/932] 10.222.0.33 () {56 vars in 1126 bytes} [Thu Sep 1 14:46:48 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:46:49.046470 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb638c9-5d79-407a-bc4d-14e936152f92 None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:49.083001 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3301e0-f254-4908-a60e-8105049c4f55 None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:49.126719 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce056de-5173-47bd-aa21-dfef163c195e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:49.127369 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce056de-5173-47bd-aa21-dfef163c195e None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:49.127640 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce056de-5173-47bd-aa21-dfef163c195e None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:49.127905 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce056de-5173-47bd-aa21-dfef163c195e None admin] PATH_INFO: `/v3/domains` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:49.129833 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce056de-5173-47bd-aa21-dfef163c195e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:46:49.129833 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce056de-5173-47bd-aa21-dfef163c195e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:49.130212 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce056de-5173-47bd-aa21-dfef163c195e None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:49.143857 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 474/933] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Sep 1 14:46:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:46:49.145412 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53681a23-9227-451b-9e2a-967703368c03 None None] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:49.145696 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53681a23-9227-451b-9e2a-967703368c03 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:49.145966 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53681a23-9227-451b-9e2a-967703368c03 None None] PATH_INFO: `/` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:49.147218 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 475/934] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 1 14:46:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:46:49.149209 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53681a23-9227-451b-9e2a-967703368c03 None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:49.171653 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb638c9-5d79-407a-bc4d-14e936152f92 None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:49.174274 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:49.200275 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:49.210178 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2051457c-fba5-47d8-a2b4-81f7212858b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:49.210813 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2051457c-fba5-47d8-a2b4-81f7212858b8 None admin] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:49.211093 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2051457c-fba5-47d8-a2b4-81f7212858b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:49.211365 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2051457c-fba5-47d8-a2b4-81f7212858b8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:49.212033 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2051457c-fba5-47d8-a2b4-81f7212858b8 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:49.213412 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2051457c-fba5-47d8-a2b4-81f7212858b8 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:49.236180 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 460/935] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Sep 1 14:46:49 2022] POST /identity/v3/endpoints => generated 347 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:46:49.241002 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b853cb-8337-4abf-8124-e2ba17b194ae None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:49.241711 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b853cb-8337-4abf-8124-e2ba17b194ae None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:49.242621 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b853cb-8337-4abf-8124-e2ba17b194ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:49.265116 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-d3b853cb-8337-4abf-8124-e2ba17b194ae None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:46:49.267583 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-d3b853cb-8337-4abf-8124-e2ba17b194ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:46:49.286653 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53681a23-9227-451b-9e2a-967703368c03 None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:49.341104 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62c09be-cc55-46a0-bdda-fed6d232c7d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:49.342482 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62c09be-cc55-46a0-bdda-fed6d232c7d8 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:49.342978 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62c09be-cc55-46a0-bdda-fed6d232c7d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:49.343611 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62c09be-cc55-46a0-bdda-fed6d232c7d8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:49.354669 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62c09be-cc55-46a0-bdda-fed6d232c7d8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:49.356476 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62c09be-cc55-46a0-bdda-fed6d232c7d8 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:49.357197 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 461/936] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 1 14:46:49 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 14:46:49.369637 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application [None req-d62c09be-cc55-46a0-bdda-fed6d232c7d8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 14:46:49.369637 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:46:49.369637 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:46:49.369637 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:46:49.369637 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:46:49.369637 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:46:49.369637 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:46:49.369637 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:46:49.369637 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:46:49.369637 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:46:49.369637 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:46:49.369637 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:46:49.369637 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 14:46:49.369637 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 14:46:49.369637 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 14:46:49.370645 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 14:46:49.370645 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:46:49.370645 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:46:49.370645 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:46:49.370645 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:46:49.370645 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:46:49.370645 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:46:49.370645 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:46:49.370645 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application with Lock( Sep 01 14:46:49.370645 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:46:49.370645 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:46:49.370645 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:46:49.370645 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:46:49.370645 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:46:49.370645 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:46:49.370645 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:46:49.370645 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:46:49.371573 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 14:46:49.371573 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 14:46:49.371573 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 14:46:49.371573 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 14:46:49.371573 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 14:46:49.371573 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 14:46:49.371573 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 14:46:49.371573 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Sep 01 14:46:49.373903 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 476/937] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Sep 1 14:46:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 14:46:49.379539 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b853cb-8337-4abf-8124-e2ba17b194ae None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:49.478589 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:49.491100 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b853cb-8337-4abf-8124-e2ba17b194ae None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:49.527159 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f90df20-c844-4335-8e68-34ffead8c42c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:49.527772 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f90df20-c844-4335-8e68-34ffead8c42c None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:49.528048 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f90df20-c844-4335-8e68-34ffead8c42c None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:49.528310 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f90df20-c844-4335-8e68-34ffead8c42c None admin] PATH_INFO: `/v3/projects` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:49.529009 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f90df20-c844-4335-8e68-34ffead8c42c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81341) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:46:49.529311 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f90df20-c844-4335-8e68-34ffead8c42c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:49.530768 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f90df20-c844-4335-8e68-34ffead8c42c None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:49.544038 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 462/938] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Sep 1 14:46:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:46:49.549902 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62c09be-cc55-46a0-bdda-fed6d232c7d8 None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:49.625402 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f90df20-c844-4335-8e68-34ffead8c42c None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:49.660681 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:49.666881 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62c09be-cc55-46a0-bdda-fed6d232c7d8 None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:49.703923 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388fd49b-c803-45a0-9965-90492b1e5599 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:49.704633 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388fd49b-c803-45a0-9965-90492b1e5599 None admin] REQUEST_METHOD: `PUT` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:49.704912 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388fd49b-c803-45a0-9965-90492b1e5599 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:49.705181 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388fd49b-c803-45a0-9965-90492b1e5599 None admin] PATH_INFO: `/v3/projects/e8c85b4bd17e4e89b56bf0f831b474d5/users/24b6f275ffb04ffeae3c2097bfd040f5/roles/050916e5d27c476c9eed7850b136dd98` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:49.707942 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388fd49b-c803-45a0-9965-90492b1e5599 None admin] RBAC: Authorizing `identity:create_grant(project_id=e8c85b4bd17e4e89b56bf0f831b474d5, user_id=24b6f275ffb04ffeae3c2097bfd040f5, role_id=050916e5d27c476c9eed7850b136dd98)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:49.709466 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388fd49b-c803-45a0-9965-90492b1e5599 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:49.727115 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 477/939] 10.222.0.33 () {58 vars in 1439 bytes} [Thu Sep 1 14:46:49 2022] PUT /identity/v3/projects/e8c85b4bd17e4e89b56bf0f831b474d5/users/24b6f275ffb04ffeae3c2097bfd040f5/roles/050916e5d27c476c9eed7850b136dd98 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:46:49.765137 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f90df20-c844-4335-8e68-34ffead8c42c None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:49.804771 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0074daa9-6044-4280-838b-f66baa92f91f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:49.805713 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0074daa9-6044-4280-838b-f66baa92f91f None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:49.806188 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0074daa9-6044-4280-838b-f66baa92f91f None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:49.806662 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0074daa9-6044-4280-838b-f66baa92f91f None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:49.807409 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0074daa9-6044-4280-838b-f66baa92f91f None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:49.808959 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0074daa9-6044-4280-838b-f66baa92f91f None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:49.820644 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application [None req-0074daa9-6044-4280-838b-f66baa92f91f None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 14:46:49.820644 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:46:49.820644 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 14:46:49.820644 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 14:46:49.820644 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 14:46:49.820644 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 14:46:49.820644 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 14:46:49.820644 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 14:46:49.820644 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 14:46:49.820644 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Sep 01 14:46:49.820644 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 14:46:49.820644 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Sep 01 14:46:49.820644 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:46:49.820644 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:46:49.820644 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:46:49.820644 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:46:49.820644 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:46:49.821994 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:46:49.821994 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:46:49.821994 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:46:49.821994 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:46:49.821994 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:46:49.821994 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:46:49.821994 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 14:46:49.821994 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 14:46:49.821994 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 14:46:49.821994 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 14:46:49.821994 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:46:49.821994 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:46:49.821994 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:46:49.821994 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:46:49.821994 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:46:49.821994 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:46:49.823090 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:46:49.823090 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application with Lock( Sep 01 14:46:49.823090 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:46:49.823090 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:46:49.823090 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:46:49.823090 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:46:49.823090 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:46:49.823090 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:46:49.823090 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:46:49.823090 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:46:49.823090 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 14:46:49.823090 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 14:46:49.823090 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 14:46:49.823090 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Sep 01 14:46:49.825693 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 463/940] 10.222.0.33 () {56 vars in 1126 bytes} [Thu Sep 1 14:46:49 2022] GET /identity/v3/services/object-store => generated 93 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 14:46:49.832771 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388fd49b-c803-45a0-9965-90492b1e5599 None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:49.945017 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388fd49b-c803-45a0-9965-90492b1e5599 None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:49.981010 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202e5c1d-2d15-4474-a5ca-997456d2ea29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:49.981913 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202e5c1d-2d15-4474-a5ca-997456d2ea29 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:49.982398 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202e5c1d-2d15-4474-a5ca-997456d2ea29 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:49.982882 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202e5c1d-2d15-4474-a5ca-997456d2ea29 None admin] PATH_INFO: `/v3/services` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:49.983697 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202e5c1d-2d15-4474-a5ca-997456d2ea29 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:46:49.984172 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202e5c1d-2d15-4474-a5ca-997456d2ea29 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:49.985716 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202e5c1d-2d15-4474-a5ca-997456d2ea29 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:49.991168 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 478/941] 10.222.0.33 () {56 vars in 1122 bytes} [Thu Sep 1 14:46:49 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:46:49.997081 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0074daa9-6044-4280-838b-f66baa92f91f None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:50.014838 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:50.107452 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0074daa9-6044-4280-838b-f66baa92f91f None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:50.147631 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bc5c45-4f03-4626-b757-dbfc9a4edc7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:50.148237 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bc5c45-4f03-4626-b757-dbfc9a4edc7e None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:50.148510 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bc5c45-4f03-4626-b757-dbfc9a4edc7e None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:50.148778 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bc5c45-4f03-4626-b757-dbfc9a4edc7e None admin] PATH_INFO: `/v3/services` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:50.149400 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4bc5c45-4f03-4626-b757-dbfc9a4edc7e None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81341) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:46:50.149694 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4bc5c45-4f03-4626-b757-dbfc9a4edc7e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:50.151105 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4bc5c45-4f03-4626-b757-dbfc9a4edc7e None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:50.156300 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 464/942] 10.222.0.33 () {56 vars in 1122 bytes} [Thu Sep 1 14:46:49 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:46:50.162199 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202e5c1d-2d15-4474-a5ca-997456d2ea29 None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:50.187532 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:50.270473 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202e5c1d-2d15-4474-a5ca-997456d2ea29 None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:50.304592 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88225f91-7841-482a-9fee-0a8b43873cf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:50.305188 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88225f91-7841-482a-9fee-0a8b43873cf7 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:50.305460 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88225f91-7841-482a-9fee-0a8b43873cf7 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:50.305725 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88225f91-7841-482a-9fee-0a8b43873cf7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:50.306444 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88225f91-7841-482a-9fee-0a8b43873cf7 None admin] RBAC: Adding query filter params (interface=public, service_id=b17f329e05384e1f9f4c29786d47f5cf, region_id=RegionOne) {{(pid=81342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:46:50.306747 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88225f91-7841-482a-9fee-0a8b43873cf7 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:50.308107 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88225f91-7841-482a-9fee-0a8b43873cf7 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:50.313561 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 479/943] 10.222.0.33 () {56 vars in 1251 bytes} [Thu Sep 1 14:46:50 2022] GET /identity/v3/endpoints?service_id=b17f329e05384e1f9f4c29786d47f5cf&interface=public®ion_id=RegionOne => generated 194 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:46:50.319717 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bc5c45-4f03-4626-b757-dbfc9a4edc7e None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:50.406713 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:50.427178 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bc5c45-4f03-4626-b757-dbfc9a4edc7e None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:50.462534 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c32e383-f622-4a6b-b7dc-0dacd156581d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:50.463136 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c32e383-f622-4a6b-b7dc-0dacd156581d None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:50.463409 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c32e383-f622-4a6b-b7dc-0dacd156581d None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:50.463678 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c32e383-f622-4a6b-b7dc-0dacd156581d None admin] PATH_INFO: `/v3/services` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:50.464274 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c32e383-f622-4a6b-b7dc-0dacd156581d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:50.465651 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c32e383-f622-4a6b-b7dc-0dacd156581d None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:50.471978 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 465/944] 10.222.0.33 () {56 vars in 1087 bytes} [Thu Sep 1 14:46:50 2022] GET /identity/v3/services => generated 1497 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:46:50.949160 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c7558d-30de-4775-baad-d710858e12e2 None None] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:50.949698 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c7558d-30de-4775-baad-d710858e12e2 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:50.949698 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c7558d-30de-4775-baad-d710858e12e2 None None] PATH_INFO: `/` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:50.951262 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 480/945] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 1 14:46:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:46:50.956419 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc28004-5be7-4d42-84fb-dc192a61f761 None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:50.957375 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc28004-5be7-4d42-84fb-dc192a61f761 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:50.957550 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc28004-5be7-4d42-84fb-dc192a61f761 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:50.976345 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-9dc28004-5be7-4d42-84fb-dc192a61f761 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:46:50.977557 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-9dc28004-5be7-4d42-84fb-dc192a61f761 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:46:51.046251 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 466/946] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 1 14:46:50 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 14:46:51.320617 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c7558d-30de-4775-baad-d710858e12e2 None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:51.368308 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2612d1-9b07-4ec4-a92a-56eddd14f790 None None] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:51.368651 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2612d1-9b07-4ec4-a92a-56eddd14f790 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:51.368857 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2612d1-9b07-4ec4-a92a-56eddd14f790 None None] PATH_INFO: `/` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:51.370746 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 467/947] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 1 14:46:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:46:51.375288 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3aa592-c8d6-4e8e-9267-d31b4ff98f01 None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:51.375566 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3aa592-c8d6-4e8e-9267-d31b4ff98f01 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:51.375836 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3aa592-c8d6-4e8e-9267-d31b4ff98f01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:51.394211 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-8f3aa592-c8d6-4e8e-9267-d31b4ff98f01 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:46:51.396137 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-8f3aa592-c8d6-4e8e-9267-d31b4ff98f01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:46:51.422601 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:51.431639 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c7558d-30de-4775-baad-d710858e12e2 None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:51.469641 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f57379-582a-465f-835a-bee2720485c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:51.470388 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f57379-582a-465f-835a-bee2720485c1 None admin] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:51.470670 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f57379-582a-465f-835a-bee2720485c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:51.470948 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f57379-582a-465f-835a-bee2720485c1 None admin] PATH_INFO: `/v3/projects` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:51.471607 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f57379-582a-465f-835a-bee2720485c1 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:51.473360 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f57379-582a-465f-835a-bee2720485c1 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:51.476331 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: WARNING keystone.server.flask.common [None req-d4f57379-582a-465f-835a-bee2720485c1 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Sep 01 14:46:51.478205 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 468/948] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 1 14:46:51 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 14:46:51.497322 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.sql.core [None req-d4f57379-582a-465f-835a-bee2720485c1 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 01 14:46:51.497322 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 01 14:46:51.497322 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [parameters: {'id': '4984397739d14e9b93d58298dc71b545', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 01 14:46:51.497322 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81342) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 01 14:46:51.515170 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application [None req-d4f57379-582a-465f-835a-bee2720485c1 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 4984397739d14e9b93d58298dc71b545.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 4984397739d14e9b93d58298dc71b545. Sep 01 14:46:51.515170 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:46:51.515170 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 01 14:46:51.515170 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 01 14:46:51.515170 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 01 14:46:51.515170 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 01 14:46:51.515170 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 01 14:46:51.515170 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application result = self._query(query) Sep 01 14:46:51.515170 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 01 14:46:51.515170 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application conn.query(q) Sep 01 14:46:51.515170 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 01 14:46:51.515170 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 01 14:46:51.515170 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 01 14:46:51.516050 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application result.read() Sep 01 14:46:51.516050 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 01 14:46:51.516050 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 01 14:46:51.516050 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 01 14:46:51.516050 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 01 14:46:51.516050 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 01 14:46:51.516050 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 01 14:46:51.516050 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 01 14:46:51.516050 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 01 14:46:51.516050 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 01 14:46:51.516050 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Sep 01 14:46:51.516050 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 01 14:46:51.516050 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Sep 01 14:46:51.516050 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:46:51.516050 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 01 14:46:51.516050 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 01 14:46:51.516050 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Sep 01 14:46:51.516050 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return project_ref.to_dict() Sep 01 14:46:51.517048 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 01 14:46:51.517048 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application next(self.gen) Sep 01 14:46:51.517048 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 01 14:46:51.517048 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application yield resource Sep 01 14:46:51.517048 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 01 14:46:51.517048 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application next(self.gen) Sep 01 14:46:51.517048 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 01 14:46:51.517048 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application self.session.rollback() Sep 01 14:46:51.517048 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 01 14:46:51.517048 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application self.force_reraise() Sep 01 14:46:51.517048 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 01 14:46:51.517048 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application raise self.value Sep 01 14:46:51.517048 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 01 14:46:51.517048 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 01 14:46:51.517048 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 01 14:46:51.517048 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application session.commit() Sep 01 14:46:51.517048 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 01 14:46:51.517958 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 01 14:46:51.517958 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 01 14:46:51.517958 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application self._prepare_impl() Sep 01 14:46:51.517958 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 01 14:46:51.517958 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application self.session.flush() Sep 01 14:46:51.517958 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 01 14:46:51.517958 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application self._flush(objects) Sep 01 14:46:51.517958 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 01 14:46:51.517958 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 01 14:46:51.517958 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 01 14:46:51.517958 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application compat.raise_( Sep 01 14:46:51.517958 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 01 14:46:51.517958 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application raise exception Sep 01 14:46:51.517958 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 01 14:46:51.517958 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application flush_context.execute() Sep 01 14:46:51.517958 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 01 14:46:51.517958 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application rec.execute(self) Sep 01 14:46:51.518923 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 01 14:46:51.518923 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 01 14:46:51.518923 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 01 14:46:51.518923 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 01 14:46:51.518923 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 01 14:46:51.518923 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 01 14:46:51.518923 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 01 14:46:51.518923 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 01 14:46:51.518923 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 01 14:46:51.518923 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 01 14:46:51.518923 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 01 14:46:51.518923 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 01 14:46:51.518923 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 01 14:46:51.518923 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 01 14:46:51.518923 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 01 14:46:51.520310 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 01 14:46:51.520310 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 01 14:46:51.520310 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application raise exception Sep 01 14:46:51.520310 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 01 14:46:51.520310 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 01 14:46:51.520310 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 01 14:46:51.520310 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 01 14:46:51.520310 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 01 14:46:51.520310 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application result = self._query(query) Sep 01 14:46:51.520310 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 01 14:46:51.520310 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application conn.query(q) Sep 01 14:46:51.520310 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 01 14:46:51.520310 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 01 14:46:51.520310 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 01 14:46:51.520310 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application result.read() Sep 01 14:46:51.520310 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 01 14:46:51.520310 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 01 14:46:51.521209 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 01 14:46:51.521209 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 01 14:46:51.521209 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 01 14:46:51.521209 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 01 14:46:51.521209 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 01 14:46:51.521209 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 01 14:46:51.521209 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 01 14:46:51.521209 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 01 14:46:51.521209 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application [parameters: {'id': '4984397739d14e9b93d58298dc71b545', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 01 14:46:51.521209 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 01 14:46:51.521209 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Sep 01 14:46:51.521209 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 14:46:51.521209 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Sep 01 14:46:51.521209 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:46:51.521209 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Sep 01 14:46:51.526342 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Sep 01 14:46:51.526342 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 01 14:46:51.526342 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 01 14:46:51.526342 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Sep 01 14:46:51.526342 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Sep 01 14:46:51.526342 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 14:46:51.526342 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Sep 01 14:46:51.526342 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:46:51.526342 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:46:51.526342 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:46:51.526342 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:46:51.526342 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:46:51.526342 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:46:51.526342 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:46:51.526342 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:46:51.526342 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:46:51.526342 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:46:51.528334 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:46:51.528334 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Sep 01 14:46:51.528334 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Sep 01 14:46:51.528334 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:46:51.528334 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:46:51.528334 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Sep 01 14:46:51.528334 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application raise exception.Conflict( Sep 01 14:46:51.528334 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 4984397739d14e9b93d58298dc71b545. Sep 01 14:46:51.528334 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Sep 01 14:46:51.528334 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 481/949] 10.222.0.33 () {60 vars in 1140 bytes} [Thu Sep 1 14:46:51 2022] POST /identity/v3/projects => generated 262 bytes in 197 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:46:51.529366 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3aa592-c8d6-4e8e-9267-d31b4ff98f01 None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:51.529366 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:51.628082 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3aa592-c8d6-4e8e-9267-d31b4ff98f01 None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:51.669650 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a52821e-4d32-45bb-be00-060778b3628f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:51.670318 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a52821e-4d32-45bb-be00-060778b3628f None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:51.670598 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a52821e-4d32-45bb-be00-060778b3628f None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:51.670867 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a52821e-4d32-45bb-be00-060778b3628f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:51.677836 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a52821e-4d32-45bb-be00-060778b3628f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:51.679317 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a52821e-4d32-45bb-be00-060778b3628f None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:51.690616 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application [None req-0a52821e-4d32-45bb-be00-060778b3628f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 14:46:51.690616 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:46:51.690616 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:46:51.690616 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:46:51.690616 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:46:51.690616 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:46:51.690616 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:46:51.690616 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:46:51.690616 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:46:51.690616 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:46:51.690616 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:46:51.690616 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:46:51.690616 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 14:46:51.690616 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 14:46:51.690616 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 14:46:51.693119 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 14:46:51.693119 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:46:51.693119 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:46:51.693119 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:46:51.693119 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:46:51.693119 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:46:51.693119 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:46:51.693119 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:46:51.693119 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application with Lock( Sep 01 14:46:51.693119 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:46:51.693119 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:46:51.693119 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:46:51.693119 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:46:51.693119 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:46:51.693119 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:46:51.693119 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:46:51.693119 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:46:51.694501 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 14:46:51.694501 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 14:46:51.694501 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 14:46:51.694501 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 14:46:51.694501 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 14:46:51.694501 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 14:46:51.694501 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 14:46:51.694501 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Sep 01 14:46:51.694501 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 469/950] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Sep 1 14:46:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:46:51.699520 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f57379-582a-465f-835a-bee2720485c1 None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:51.728846 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a669cf-e43a-4a18-9ec1-a228087ed122 None None] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:51.729177 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a669cf-e43a-4a18-9ec1-a228087ed122 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:51.729390 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a669cf-e43a-4a18-9ec1-a228087ed122 None None] PATH_INFO: `/` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:51.731039 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 470/951] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 1 14:46:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 14:46:51.736899 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc2c9bd-52ac-40b4-8234-a635440e910c None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:51.737180 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc2c9bd-52ac-40b4-8234-a635440e910c None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:51.737452 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc2c9bd-52ac-40b4-8234-a635440e910c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:51.761029 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-6bc2c9bd-52ac-40b4-8234-a635440e910c None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:46:51.762320 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-6bc2c9bd-52ac-40b4-8234-a635440e910c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:46:51.812976 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f57379-582a-465f-835a-bee2720485c1 None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:51.827509 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:51.838912 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 471/952] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 1 14:46:51 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 14:46:51.847206 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc2c9bd-52ac-40b4-8234-a635440e910c None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:51.849919 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d42d9d-d277-43f1-9ea4-017074fc6a79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:51.850558 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d42d9d-d277-43f1-9ea4-017074fc6a79 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:51.850837 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d42d9d-d277-43f1-9ea4-017074fc6a79 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:51.851107 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d42d9d-d277-43f1-9ea4-017074fc6a79 None admin] PATH_INFO: `/v3/projects` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:51.852360 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d42d9d-d277-43f1-9ea4-017074fc6a79 None admin] RBAC: Adding query filter params (name=service) {{(pid=81342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:46:51.852360 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d42d9d-d277-43f1-9ea4-017074fc6a79 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:51.853728 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d42d9d-d277-43f1-9ea4-017074fc6a79 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:51.879616 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 482/953] 10.222.0.33 () {56 vars in 1112 bytes} [Thu Sep 1 14:46:51 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:46:51.966709 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc2c9bd-52ac-40b4-8234-a635440e910c None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:52.003411 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df829c53-589e-442b-86e0-e1624e9e7f48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:52.004636 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df829c53-589e-442b-86e0-e1624e9e7f48 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:52.004911 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df829c53-589e-442b-86e0-e1624e9e7f48 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:52.005178 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df829c53-589e-442b-86e0-e1624e9e7f48 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:52.005765 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df829c53-589e-442b-86e0-e1624e9e7f48 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:52.007186 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df829c53-589e-442b-86e0-e1624e9e7f48 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:52.016901 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application [None req-df829c53-589e-442b-86e0-e1624e9e7f48 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 01 14:46:52.016901 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:46:52.016901 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 14:46:52.016901 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 14:46:52.016901 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 14:46:52.016901 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 14:46:52.016901 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 14:46:52.016901 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 14:46:52.016901 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 01 14:46:52.016901 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Sep 01 14:46:52.016901 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 14:46:52.016901 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Sep 01 14:46:52.016901 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:46:52.016901 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:46:52.016901 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:46:52.016901 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:46:52.016901 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:46:52.018012 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:46:52.018012 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:46:52.018012 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:46:52.018012 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:46:52.018012 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:46:52.018012 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:46:52.018012 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 14:46:52.018012 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 14:46:52.018012 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 14:46:52.018012 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 14:46:52.018012 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:46:52.018012 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:46:52.018012 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:46:52.018012 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:46:52.018012 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:46:52.018012 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:46:52.019125 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:46:52.019125 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application with Lock( Sep 01 14:46:52.019125 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:46:52.019125 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:46:52.019125 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:46:52.019125 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:46:52.019125 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:46:52.019125 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:46:52.019125 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:46:52.019125 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:46:52.019125 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 14:46:52.019125 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 14:46:52.019125 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 01 14:46:52.019125 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Sep 01 14:46:52.019125 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 472/954] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Sep 1 14:46:51 2022] GET /identity/v3/services/compute => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:46:52.031445 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d42d9d-d277-43f1-9ea4-017074fc6a79 None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:52.034541 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:52.103939 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df829c53-589e-442b-86e0-e1624e9e7f48 None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:52.155730 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d42d9d-d277-43f1-9ea4-017074fc6a79 None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:52.166392 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:52.198215 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8b90e4-b6e1-473b-a0de-776417878f86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:52.198845 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8b90e4-b6e1-473b-a0de-776417878f86 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:52.199116 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8b90e4-b6e1-473b-a0de-776417878f86 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:52.199382 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8b90e4-b6e1-473b-a0de-776417878f86 None admin] PATH_INFO: `/v3/services` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:52.200005 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8b90e4-b6e1-473b-a0de-776417878f86 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:46:52.200298 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8b90e4-b6e1-473b-a0de-776417878f86 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:52.201648 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8b90e4-b6e1-473b-a0de-776417878f86 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:52.213303 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 483/955] 10.222.0.33 () {56 vars in 1112 bytes} [Thu Sep 1 14:46:52 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:46:52.218948 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8b90e4-b6e1-473b-a0de-776417878f86 None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:52.234426 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df829c53-589e-442b-86e0-e1624e9e7f48 None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:52.280171 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba3d8f8-b436-4214-bbe2-411923045518 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:52.281082 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba3d8f8-b436-4214-bbe2-411923045518 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:52.281362 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba3d8f8-b436-4214-bbe2-411923045518 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:52.281873 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba3d8f8-b436-4214-bbe2-411923045518 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:52.292513 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:52.300032 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba3d8f8-b436-4214-bbe2-411923045518 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 14:46:52.306547 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application [None req-7ba3d8f8-b436-4214-bbe2-411923045518 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 01 14:46:52.306547 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:46:52.306547 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:46:52.306547 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:46:52.306547 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:46:52.306547 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:46:52.306547 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:46:52.306547 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:46:52.306547 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:46:52.306547 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:46:52.306547 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:46:52.306547 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:46:52.306547 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 14:46:52.306547 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 14:46:52.306547 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 14:46:52.306547 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 14:46:52.307711 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 14:46:52.307711 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 14:46:52.307711 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 14:46:52.307711 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 14:46:52.307711 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:46:52.307711 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:46:52.307711 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:46:52.307711 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:46:52.307711 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:46:52.307711 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:46:52.307711 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:46:52.307711 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application with Lock( Sep 01 14:46:52.307711 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:46:52.307711 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:46:52.307711 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:46:52.307711 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:46:52.308600 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:46:52.308600 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:46:52.308600 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:46:52.308600 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:46:52.308600 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 14:46:52.308600 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 14:46:52.308600 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 14:46:52.308600 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 14:46:52.308600 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 14:46:52.308600 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 14:46:52.308600 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 01 14:46:52.308600 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Sep 01 14:46:52.310332 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 473/956] 10.222.0.33 () {56 vars in 1120 bytes} [Thu Sep 1 14:46:52 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:46:52.314075 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba3d8f8-b436-4214-bbe2-411923045518 None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:52.338617 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8b90e4-b6e1-473b-a0de-776417878f86 None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:52.375775 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b523bb2c-1a74-461b-94d4-1258abcc44de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:52.376186 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b523bb2c-1a74-461b-94d4-1258abcc44de None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:52.376465 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b523bb2c-1a74-461b-94d4-1258abcc44de None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:52.376728 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b523bb2c-1a74-461b-94d4-1258abcc44de None admin] PATH_INFO: `/v3/services` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:52.377354 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b523bb2c-1a74-461b-94d4-1258abcc44de None admin] RBAC: Adding query filter params (type=compute) {{(pid=81342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:46:52.377651 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b523bb2c-1a74-461b-94d4-1258abcc44de None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:52.379116 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b523bb2c-1a74-461b-94d4-1258abcc44de None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:52.390386 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 484/957] 10.222.0.33 () {56 vars in 1112 bytes} [Thu Sep 1 14:46:52 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 14:46:52.432125 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba3d8f8-b436-4214-bbe2-411923045518 None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:52.452114 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:52.474217 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bcc344-7bd4-45d6-8155-16be4c2ee4c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:52.475245 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bcc344-7bd4-45d6-8155-16be4c2ee4c9 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:52.475732 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bcc344-7bd4-45d6-8155-16be4c2ee4c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:52.476208 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bcc344-7bd4-45d6-8155-16be4c2ee4c9 None admin] PATH_INFO: `/v3/roles` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:52.477114 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6bcc344-7bd4-45d6-8155-16be4c2ee4c9 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81341) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:46:52.477618 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6bcc344-7bd4-45d6-8155-16be4c2ee4c9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:52.479348 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6bcc344-7bd4-45d6-8155-16be4c2ee4c9 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:52.490216 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d2f5a7-d8f3-4702-b268-08717e0cc447 None None] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:52.490216 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d2f5a7-d8f3-4702-b268-08717e0cc447 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:52.490216 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d2f5a7-d8f3-4702-b268-08717e0cc447 None None] PATH_INFO: `/` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:52.490216 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 485/958] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 1 14:46:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 14:46:52.490753 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 474/959] 10.222.0.33 () {56 vars in 1115 bytes} [Thu Sep 1 14:46:52 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 14:46:52.491183 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c327f9a-e813-49ef-bb6e-fc94e5237fa0 None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:52.491470 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c327f9a-e813-49ef-bb6e-fc94e5237fa0 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:52.491745 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c327f9a-e813-49ef-bb6e-fc94e5237fa0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:52.497921 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bcc344-7bd4-45d6-8155-16be4c2ee4c9 None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:52.516056 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-4c327f9a-e813-49ef-bb6e-fc94e5237fa0 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:46:52.517932 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-4c327f9a-e813-49ef-bb6e-fc94e5237fa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:46:52.616971 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 486/960] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 1 14:46:52 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 14:46:52.620242 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bcc344-7bd4-45d6-8155-16be4c2ee4c9 None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:52.657522 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9431a6d-5525-40dd-a072-7a8dc0112feb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:52.658188 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9431a6d-5525-40dd-a072-7a8dc0112feb None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:52.658565 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9431a6d-5525-40dd-a072-7a8dc0112feb None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:52.658835 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9431a6d-5525-40dd-a072-7a8dc0112feb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:52.671035 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9431a6d-5525-40dd-a072-7a8dc0112feb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:52.672606 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9431a6d-5525-40dd-a072-7a8dc0112feb None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:52.687260 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application [None req-d9431a6d-5525-40dd-a072-7a8dc0112feb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 14:46:52.687260 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:46:52.687260 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:46:52.687260 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:46:52.687260 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:46:52.687260 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:46:52.687260 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:46:52.687260 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:46:52.687260 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:46:52.687260 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:46:52.687260 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:46:52.687260 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:46:52.687260 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 14:46:52.687260 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 14:46:52.687260 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 14:46:52.687260 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 14:46:52.688281 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:46:52.688281 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:46:52.688281 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:46:52.688281 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:46:52.688281 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:46:52.688281 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:46:52.688281 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:46:52.688281 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application with Lock( Sep 01 14:46:52.688281 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:46:52.688281 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:46:52.688281 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:46:52.688281 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:46:52.688281 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:46:52.688281 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:46:52.688281 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:46:52.688281 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:46:52.688281 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 14:46:52.689228 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 14:46:52.689228 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 14:46:52.689228 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Sep 01 14:46:52.689228 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 475/961] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 1 14:46:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 14:46:52.695983 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c327f9a-e813-49ef-bb6e-fc94e5237fa0 None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:52.758455 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:52.801562 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c327f9a-e813-49ef-bb6e-fc94e5237fa0 None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:52.837704 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c460888a-117b-40fe-95ce-3e70f5fd1f28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:52.838698 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c460888a-117b-40fe-95ce-3e70f5fd1f28 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:52.839194 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c460888a-117b-40fe-95ce-3e70f5fd1f28 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:52.839677 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c460888a-117b-40fe-95ce-3e70f5fd1f28 None admin] PATH_INFO: `/v3/domains` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:52.840519 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c460888a-117b-40fe-95ce-3e70f5fd1f28 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:46:52.841024 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c460888a-117b-40fe-95ce-3e70f5fd1f28 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:52.842684 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c460888a-117b-40fe-95ce-3e70f5fd1f28 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:52.854688 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 487/962] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Sep 1 14:46:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:46:52.860546 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9431a6d-5525-40dd-a072-7a8dc0112feb None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:52.891152 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c460888a-117b-40fe-95ce-3e70f5fd1f28 None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:52.895676 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:52.984002 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9431a6d-5525-40dd-a072-7a8dc0112feb None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:53.016927 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c460888a-117b-40fe-95ce-3e70f5fd1f28 None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:53.018702 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f3919e-dd71-4346-ba2e-e78e09aac08e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:53.019375 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f3919e-dd71-4346-ba2e-e78e09aac08e None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:53.019650 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f3919e-dd71-4346-ba2e-e78e09aac08e None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:53.019916 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f3919e-dd71-4346-ba2e-e78e09aac08e None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:53.028203 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f3919e-dd71-4346-ba2e-e78e09aac08e None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:53.029837 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f3919e-dd71-4346-ba2e-e78e09aac08e None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:53.045339 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application [None req-94f3919e-dd71-4346-ba2e-e78e09aac08e None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 01 14:46:53.045339 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:46:53.045339 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:46:53.045339 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:46:53.045339 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:46:53.045339 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:46:53.045339 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:46:53.045339 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:46:53.045339 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:46:53.045339 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:46:53.045339 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:46:53.045339 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:46:53.045339 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 14:46:53.045339 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 14:46:53.045339 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 14:46:53.045339 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 14:46:53.046451 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:46:53.046451 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:46:53.046451 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 14:46:53.046451 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 14:46:53.046451 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 14:46:53.046451 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 14:46:53.046451 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:46:53.046451 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:46:53.046451 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:46:53.046451 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:46:53.046451 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:46:53.046451 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application with Lock( Sep 01 14:46:53.046451 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:46:53.046451 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:46:53.046451 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:46:53.046451 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:46:53.046451 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:46:53.046451 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:46:53.047452 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:46:53.047452 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:46:53.047452 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 14:46:53.047452 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 14:46:53.047452 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 14:46:53.047452 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 14:46:53.047452 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 14:46:53.047452 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 14:46:53.047452 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 01 14:46:53.047452 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Sep 01 14:46:53.051790 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 476/963] 10.222.0.33 () {56 vars in 1117 bytes} [Thu Sep 1 14:46:52 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:46:53.052269 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:53.052704 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f3919e-dd71-4346-ba2e-e78e09aac08e None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:53.071261 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d3074f-3389-4578-b4f5-7121260d4976 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:53.072526 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d3074f-3389-4578-b4f5-7121260d4976 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:53.073021 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d3074f-3389-4578-b4f5-7121260d4976 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:53.073619 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d3074f-3389-4578-b4f5-7121260d4976 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:53.074774 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d3074f-3389-4578-b4f5-7121260d4976 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:53.076469 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d3074f-3389-4578-b4f5-7121260d4976 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:53.088785 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application [None req-30d3074f-3389-4578-b4f5-7121260d4976 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 14:46:53.088785 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:46:53.088785 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 14:46:53.088785 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 14:46:53.088785 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 14:46:53.088785 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 14:46:53.088785 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 14:46:53.088785 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 14:46:53.088785 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 14:46:53.088785 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Sep 01 14:46:53.088785 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 14:46:53.088785 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Sep 01 14:46:53.088785 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:46:53.088785 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:46:53.088785 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:46:53.088785 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:46:53.088785 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:46:53.089871 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:46:53.089871 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:46:53.089871 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:46:53.089871 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:46:53.089871 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:46:53.089871 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:46:53.089871 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 14:46:53.089871 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 14:46:53.089871 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 14:46:53.089871 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 14:46:53.089871 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:46:53.089871 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:46:53.089871 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:46:53.089871 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:46:53.089871 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:46:53.089871 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:46:53.090775 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:46:53.090775 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application with Lock( Sep 01 14:46:53.090775 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:46:53.090775 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:46:53.090775 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:46:53.090775 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:46:53.090775 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:46:53.090775 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:46:53.090775 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:46:53.090775 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:46:53.090775 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 14:46:53.090775 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 14:46:53.090775 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 14:46:53.090775 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Sep 01 14:46:53.096839 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 488/964] 10.222.0.33 () {56 vars in 1126 bytes} [Thu Sep 1 14:46:52 2022] GET /identity/v3/services/object-store => generated 93 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 14:46:53.098125 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d3074f-3389-4578-b4f5-7121260d4976 None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:53.163161 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:53.174217 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f3919e-dd71-4346-ba2e-e78e09aac08e None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:53.210273 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d12353f-5684-498a-a117-8825ad91b2a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:53.210607 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d12353f-5684-498a-a117-8825ad91b2a6 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:53.210898 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d12353f-5684-498a-a117-8825ad91b2a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:53.211163 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d12353f-5684-498a-a117-8825ad91b2a6 None admin] PATH_INFO: `/v3/users` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:53.211888 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d12353f-5684-498a-a117-8825ad91b2a6 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81341) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:46:53.212187 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d12353f-5684-498a-a117-8825ad91b2a6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:53.213605 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d12353f-5684-498a-a117-8825ad91b2a6 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:53.233541 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 477/965] 10.222.0.33 () {56 vars in 1149 bytes} [Thu Sep 1 14:46:53 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:46:53.239010 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d12353f-5684-498a-a117-8825ad91b2a6 None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:53.245425 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d3074f-3389-4578-b4f5-7121260d4976 None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:53.292835 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ead878e-8726-46f0-bd25-a377095cf804 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:53.293437 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ead878e-8726-46f0-bd25-a377095cf804 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:53.293709 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ead878e-8726-46f0-bd25-a377095cf804 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:53.293970 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ead878e-8726-46f0-bd25-a377095cf804 None admin] PATH_INFO: `/v3/services` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:53.294625 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ead878e-8726-46f0-bd25-a377095cf804 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:46:53.294920 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ead878e-8726-46f0-bd25-a377095cf804 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:53.296274 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ead878e-8726-46f0-bd25-a377095cf804 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:53.305846 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 489/966] 10.222.0.33 () {56 vars in 1122 bytes} [Thu Sep 1 14:46:53 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:46:53.306810 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ead878e-8726-46f0-bd25-a377095cf804 None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:53.357083 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:53.372487 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d12353f-5684-498a-a117-8825ad91b2a6 None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:53.410588 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ead878e-8726-46f0-bd25-a377095cf804 None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:53.417700 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248e6dc2-32a1-49f6-aa16-37d85598e01c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:53.418980 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248e6dc2-32a1-49f6-aa16-37d85598e01c None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:53.419445 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248e6dc2-32a1-49f6-aa16-37d85598e01c None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:53.419902 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248e6dc2-32a1-49f6-aa16-37d85598e01c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:53.439807 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:53.455235 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f67074-6532-4437-8d34-8ab3a8fa6bf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:53.456389 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f67074-6532-4437-8d34-8ab3a8fa6bf8 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:53.457396 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f67074-6532-4437-8d34-8ab3a8fa6bf8 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:53.457845 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f67074-6532-4437-8d34-8ab3a8fa6bf8 None admin] PATH_INFO: `/v3/services` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:53.458686 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f67074-6532-4437-8d34-8ab3a8fa6bf8 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:46:53.459331 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f67074-6532-4437-8d34-8ab3a8fa6bf8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:53.461723 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f67074-6532-4437-8d34-8ab3a8fa6bf8 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:53.473610 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 490/967] 10.222.0.33 () {56 vars in 1122 bytes} [Thu Sep 1 14:46:53 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:46:53.474162 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f67074-6532-4437-8d34-8ab3a8fa6bf8 None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:53.505599 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248e6dc2-32a1-49f6-aa16-37d85598e01c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:53.508300 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248e6dc2-32a1-49f6-aa16-37d85598e01c None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:53.585357 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f67074-6532-4437-8d34-8ab3a8fa6bf8 None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:53.601154 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:53.625476 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77024a09-4d30-4c0e-8604-5c015258b277 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:53.626178 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77024a09-4d30-4c0e-8604-5c015258b277 None admin] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:53.626567 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77024a09-4d30-4c0e-8604-5c015258b277 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:53.626895 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77024a09-4d30-4c0e-8604-5c015258b277 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:53.627761 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77024a09-4d30-4c0e-8604-5c015258b277 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:53.629852 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77024a09-4d30-4c0e-8604-5c015258b277 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:53.633570 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 478/968] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Sep 1 14:46:53 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 14:46:53.638868 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248e6dc2-32a1-49f6-aa16-37d85598e01c None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:53.662571 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 491/969] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Sep 1 14:46:53 2022] POST /identity/v3/endpoints => generated 349 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 14:46:53.703787 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:53.762762 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248e6dc2-32a1-49f6-aa16-37d85598e01c None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:53.807721 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9b08ff-1170-42ad-a14a-50cecb7c87e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:53.808688 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9b08ff-1170-42ad-a14a-50cecb7c87e9 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:53.809147 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9b08ff-1170-42ad-a14a-50cecb7c87e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:53.809596 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9b08ff-1170-42ad-a14a-50cecb7c87e9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:53.821298 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9b08ff-1170-42ad-a14a-50cecb7c87e9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:53.823077 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9b08ff-1170-42ad-a14a-50cecb7c87e9 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:53.839148 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application [None req-1d9b08ff-1170-42ad-a14a-50cecb7c87e9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 14:46:53.839148 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:46:53.839148 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:46:53.839148 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:46:53.839148 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:46:53.839148 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:46:53.839148 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:46:53.839148 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:46:53.839148 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:46:53.839148 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:46:53.839148 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:46:53.839148 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:46:53.839148 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 14:46:53.839148 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 14:46:53.839148 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 14:46:53.839148 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 14:46:53.840186 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:46:53.840186 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:46:53.840186 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:46:53.840186 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:46:53.840186 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:46:53.840186 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:46:53.840186 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:46:53.840186 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application with Lock( Sep 01 14:46:53.840186 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:46:53.840186 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:46:53.840186 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:46:53.840186 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:46:53.840186 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:46:53.840186 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:46:53.840186 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:46:53.840186 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:46:53.840186 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 14:46:53.841129 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 14:46:53.841129 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 14:46:53.841129 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Sep 01 14:46:53.841611 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 479/970] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 1 14:46:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 14:46:53.847163 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77024a09-4d30-4c0e-8604-5c015258b277 None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:53.954480 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77024a09-4d30-4c0e-8604-5c015258b277 None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:53.975492 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94825066-969c-4e2b-b8e8-d71721c8b5a3 None None] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:53.975842 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94825066-969c-4e2b-b8e8-d71721c8b5a3 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:53.976050 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94825066-969c-4e2b-b8e8-d71721c8b5a3 None None] PATH_INFO: `/` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:53.977652 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 480/971] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 1 14:46:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 14:46:53.982606 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2cbdc0-3d3c-441c-9e75-e192bb71a896 None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:53.982889 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2cbdc0-3d3c-441c-9e75-e192bb71a896 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:53.983157 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2cbdc0-3d3c-441c-9e75-e192bb71a896 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:53.994971 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1eb799-57ce-482b-9065-a6886207f345 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:53.995914 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1eb799-57ce-482b-9065-a6886207f345 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:53.996418 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1eb799-57ce-482b-9065-a6886207f345 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:53.997314 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1eb799-57ce-482b-9065-a6886207f345 None admin] PATH_INFO: `/v3/domains` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:53.998340 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1eb799-57ce-482b-9065-a6886207f345 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:46:53.998917 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1eb799-57ce-482b-9065-a6886207f345 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:54.000522 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1eb799-57ce-482b-9065-a6886207f345 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:54.002726 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-5e2cbdc0-3d3c-441c-9e75-e192bb71a896 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:46:54.004608 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-5e2cbdc0-3d3c-441c-9e75-e192bb71a896 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:46:54.016963 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 492/972] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Sep 1 14:46:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:46:54.023689 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1eb799-57ce-482b-9065-a6886207f345 None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:54.060409 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:54.090625 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 481/973] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 1 14:46:53 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 14:46:54.142040 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1eb799-57ce-482b-9065-a6886207f345 None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:54.177448 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cae16b4-6a3b-4f92-bc46-1198233f26ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:54.178375 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cae16b4-6a3b-4f92-bc46-1198233f26ae None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:54.178797 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cae16b4-6a3b-4f92-bc46-1198233f26ae None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:54.179223 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cae16b4-6a3b-4f92-bc46-1198233f26ae None admin] PATH_INFO: `/v3/projects/service` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:54.186206 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cae16b4-6a3b-4f92-bc46-1198233f26ae None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:54.187832 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cae16b4-6a3b-4f92-bc46-1198233f26ae None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:54.199920 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application [None req-1cae16b4-6a3b-4f92-bc46-1198233f26ae None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 14:46:54.199920 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:46:54.199920 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:46:54.199920 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:46:54.199920 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:46:54.199920 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:46:54.199920 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:46:54.199920 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:46:54.199920 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:46:54.199920 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:46:54.199920 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:46:54.199920 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:46:54.199920 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 14:46:54.199920 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 14:46:54.199920 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 14:46:54.201006 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 14:46:54.201006 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:46:54.201006 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:46:54.201006 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:46:54.201006 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:46:54.201006 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:46:54.201006 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:46:54.201006 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:46:54.201006 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application with Lock( Sep 01 14:46:54.201006 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:46:54.201006 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:46:54.201006 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:46:54.201006 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:46:54.201006 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:46:54.201006 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:46:54.201006 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:46:54.201006 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:46:54.202116 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 14:46:54.202116 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 14:46:54.202116 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 14:46:54.202116 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 14:46:54.202116 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 14:46:54.202116 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 14:46:54.202116 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 14:46:54.202116 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Sep 01 14:46:54.204098 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 493/974] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Sep 1 14:46:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:46:54.209894 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2cbdc0-3d3c-441c-9e75-e192bb71a896 None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:54.238637 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:54.312784 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2cbdc0-3d3c-441c-9e75-e192bb71a896 None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:54.348551 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18efa7c0-84a6-4dd9-a555-9b4a6dfc499b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:54.349396 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18efa7c0-84a6-4dd9-a555-9b4a6dfc499b None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:54.349836 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18efa7c0-84a6-4dd9-a555-9b4a6dfc499b None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:54.350279 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18efa7c0-84a6-4dd9-a555-9b4a6dfc499b None admin] PATH_INFO: `/v3/projects` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:54.351213 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18efa7c0-84a6-4dd9-a555-9b4a6dfc499b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81341) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:46:54.351657 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18efa7c0-84a6-4dd9-a555-9b4a6dfc499b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:54.353235 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18efa7c0-84a6-4dd9-a555-9b4a6dfc499b None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:54.361161 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cae16b4-6a3b-4f92-bc46-1198233f26ae None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:54.367916 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 482/975] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Sep 1 14:46:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:46:54.373944 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18efa7c0-84a6-4dd9-a555-9b4a6dfc499b None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:54.463568 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:54.463961 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:54.477157 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cae16b4-6a3b-4f92-bc46-1198233f26ae None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:54.481609 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18efa7c0-84a6-4dd9-a555-9b4a6dfc499b None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:54.515414 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55463bb6-ec4e-4d6f-83b3-d44d1a0513ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:54.516011 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55463bb6-ec4e-4d6f-83b3-d44d1a0513ac None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:54.516288 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55463bb6-ec4e-4d6f-83b3-d44d1a0513ac None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:54.516554 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55463bb6-ec4e-4d6f-83b3-d44d1a0513ac None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:54.517273 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55463bb6-ec4e-4d6f-83b3-d44d1a0513ac None admin] RBAC: Adding query filter params (role.id=050916e5d27c476c9eed7850b136dd98, scope.project.id=e8c85b4bd17e4e89b56bf0f831b474d5, user.id=24b6f275ffb04ffeae3c2097bfd040f5) {{(pid=81341) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:46:54.517524 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c562281c-a8cb-4628-ac58-c22d97b447c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:54.517651 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55463bb6-ec4e-4d6f-83b3-d44d1a0513ac None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:54.518472 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c562281c-a8cb-4628-ac58-c22d97b447c1 None admin] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:54.519025 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55463bb6-ec4e-4d6f-83b3-d44d1a0513ac None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:54.519164 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c562281c-a8cb-4628-ac58-c22d97b447c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:54.519627 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c562281c-a8cb-4628-ac58-c22d97b447c1 None admin] PATH_INFO: `/v3/users` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:54.520477 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c562281c-a8cb-4628-ac58-c22d97b447c1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:54.522926 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c562281c-a8cb-4628-ac58-c22d97b447c1 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:54.525082 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 483/976] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Sep 1 14:46:54 2022] GET /identity/v3/role_assignments?user.id=24b6f275ffb04ffeae3c2097bfd040f5&scope.project.id=e8c85b4bd17e4e89b56bf0f831b474d5&role.id=050916e5d27c476c9eed7850b136dd98 => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:46:54.526210 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362fe9b5-6eed-4154-ab95-657000d4bec4 None None] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:54.526522 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362fe9b5-6eed-4154-ab95-657000d4bec4 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:54.526795 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362fe9b5-6eed-4154-ab95-657000d4bec4 None None] PATH_INFO: `/` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:54.528273 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 484/977] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 1 14:46:54 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:46:54.536231 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12792491-0c55-456f-90c9-f30b17b0a692 None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:54.536513 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12792491-0c55-456f-90c9-f30b17b0a692 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:54.536782 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12792491-0c55-456f-90c9-f30b17b0a692 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:54.548114 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.sql.core [None req-c562281c-a8cb-4628-ac58-c22d97b447c1 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 01 14:46:54.548114 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 01 14:46:54.548114 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [parameters: {'user_id': '135a753992c94c22874d12afb67492e9', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 01 14:46:54.548114 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81342) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 01 14:46:54.559797 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application [None req-c562281c-a8cb-4628-ac58-c22d97b447c1 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 01 14:46:54.559797 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:46:54.559797 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 01 14:46:54.559797 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 01 14:46:54.559797 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 01 14:46:54.559797 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 01 14:46:54.559797 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 01 14:46:54.559797 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application result = self._query(query) Sep 01 14:46:54.559797 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 01 14:46:54.559797 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application conn.query(q) Sep 01 14:46:54.559797 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 01 14:46:54.559797 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 01 14:46:54.559797 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 01 14:46:54.559797 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application result.read() Sep 01 14:46:54.559797 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 01 14:46:54.560925 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 01 14:46:54.560925 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 01 14:46:54.560925 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 01 14:46:54.560925 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 01 14:46:54.560925 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 01 14:46:54.560925 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 01 14:46:54.560925 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 01 14:46:54.560925 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 01 14:46:54.560925 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Sep 01 14:46:54.560925 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 01 14:46:54.560925 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Sep 01 14:46:54.560925 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:46:54.560925 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 01 14:46:54.560925 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 01 14:46:54.560925 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Sep 01 14:46:54.560925 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Sep 01 14:46:54.560925 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 01 14:46:54.560925 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application next(self.gen) Sep 01 14:46:54.561934 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 01 14:46:54.561934 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application yield resource Sep 01 14:46:54.561934 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 01 14:46:54.561934 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application next(self.gen) Sep 01 14:46:54.561934 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 01 14:46:54.561934 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application self.session.rollback() Sep 01 14:46:54.561934 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 01 14:46:54.561934 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application self.force_reraise() Sep 01 14:46:54.561934 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 01 14:46:54.561934 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application raise self.value Sep 01 14:46:54.561934 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 01 14:46:54.561934 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 01 14:46:54.561934 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 01 14:46:54.561934 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application session.commit() Sep 01 14:46:54.561934 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 01 14:46:54.561934 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 01 14:46:54.561934 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 01 14:46:54.562947 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application self._prepare_impl() Sep 01 14:46:54.562947 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 01 14:46:54.562947 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application self.session.flush() Sep 01 14:46:54.562947 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 01 14:46:54.562947 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application self._flush(objects) Sep 01 14:46:54.562947 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 01 14:46:54.562947 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 01 14:46:54.562947 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 01 14:46:54.562947 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application compat.raise_( Sep 01 14:46:54.562947 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 01 14:46:54.562947 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application raise exception Sep 01 14:46:54.562947 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 01 14:46:54.562947 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application flush_context.execute() Sep 01 14:46:54.562947 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 01 14:46:54.562947 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application rec.execute(self) Sep 01 14:46:54.562947 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 01 14:46:54.562947 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 01 14:46:54.564177 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 01 14:46:54.564177 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 01 14:46:54.564177 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Sep 01 14:46:54.564177 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application result = connection._execute_20( Sep 01 14:46:54.564177 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 01 14:46:54.564177 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 01 14:46:54.564177 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 01 14:46:54.564177 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 01 14:46:54.564177 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 01 14:46:54.564177 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 01 14:46:54.564177 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 01 14:46:54.564177 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 01 14:46:54.564177 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 01 14:46:54.564177 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 01 14:46:54.564177 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 01 14:46:54.564177 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application raise exception Sep 01 14:46:54.565249 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 01 14:46:54.565249 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 01 14:46:54.565249 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 01 14:46:54.565249 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 01 14:46:54.565249 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 01 14:46:54.565249 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application result = self._query(query) Sep 01 14:46:54.565249 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 01 14:46:54.565249 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application conn.query(q) Sep 01 14:46:54.565249 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 01 14:46:54.565249 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 01 14:46:54.565249 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 01 14:46:54.565249 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application result.read() Sep 01 14:46:54.565249 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 01 14:46:54.565249 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 01 14:46:54.565249 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 01 14:46:54.565249 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 01 14:46:54.567210 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 01 14:46:54.567210 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 01 14:46:54.567210 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 01 14:46:54.567210 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 01 14:46:54.567210 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 01 14:46:54.567210 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 01 14:46:54.567210 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application [parameters: {'user_id': '135a753992c94c22874d12afb67492e9', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 01 14:46:54.567210 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 01 14:46:54.567210 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Sep 01 14:46:54.567210 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 14:46:54.567210 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Sep 01 14:46:54.567210 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:46:54.567210 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:46:54.567210 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:46:54.567210 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:46:54.569228 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:46:54.569228 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:46:54.569228 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:46:54.569228 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:46:54.569228 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:46:54.569228 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:46:54.569228 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:46:54.569228 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Sep 01 14:46:54.569228 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Sep 01 14:46:54.569228 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:46:54.569228 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:46:54.569228 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 14:46:54.569228 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 14:46:54.569228 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 14:46:54.569228 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 14:46:54.569228 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Sep 01 14:46:54.569228 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Sep 01 14:46:54.571708 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Sep 01 14:46:54.571708 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Sep 01 14:46:54.571708 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 01 14:46:54.571708 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 01 14:46:54.571708 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 01 14:46:54.571708 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Sep 01 14:46:54.571708 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 494/978] 10.222.0.33 () {60 vars in 1131 bytes} [Thu Sep 1 14:46:54 2022] POST /identity/v3/users => generated 160 bytes in 201 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Sep 01 14:46:54.571708 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c562281c-a8cb-4628-ac58-c22d97b447c1 None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:54.572726 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-12792491-0c55-456f-90c9-f30b17b0a692 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:46:54.572726 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-12792491-0c55-456f-90c9-f30b17b0a692 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:46:54.666435 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 485/979] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 1 14:46:54 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 14:46:54.697381 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c562281c-a8cb-4628-ac58-c22d97b447c1 None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:54.735597 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040e70a0-f28e-4683-b6dd-eb5b0b3a869e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:54.736235 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040e70a0-f28e-4683-b6dd-eb5b0b3a869e None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:54.736517 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040e70a0-f28e-4683-b6dd-eb5b0b3a869e None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:54.736788 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040e70a0-f28e-4683-b6dd-eb5b0b3a869e None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:54.744515 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040e70a0-f28e-4683-b6dd-eb5b0b3a869e None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:54.746217 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040e70a0-f28e-4683-b6dd-eb5b0b3a869e None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:54.758666 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application [None req-040e70a0-f28e-4683-b6dd-eb5b0b3a869e None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 01 14:46:54.758666 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:46:54.758666 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:46:54.758666 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:46:54.758666 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:46:54.758666 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:46:54.758666 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:46:54.758666 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:46:54.758666 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:46:54.758666 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:46:54.758666 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:46:54.758666 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:46:54.758666 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 14:46:54.758666 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 14:46:54.758666 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 14:46:54.758666 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 14:46:54.759705 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:46:54.759705 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:46:54.759705 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 14:46:54.759705 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 14:46:54.759705 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 14:46:54.759705 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 14:46:54.759705 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:46:54.759705 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:46:54.759705 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:46:54.759705 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:46:54.759705 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:46:54.759705 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application with Lock( Sep 01 14:46:54.759705 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:46:54.759705 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:46:54.759705 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:46:54.759705 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:46:54.759705 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:46:54.759705 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:46:54.760738 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:46:54.760738 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:46:54.760738 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 14:46:54.760738 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 14:46:54.760738 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 14:46:54.760738 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 14:46:54.760738 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 14:46:54.760738 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 14:46:54.760738 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 01 14:46:54.760738 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Sep 01 14:46:54.760738 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 495/980] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Sep 1 14:46:54 2022] GET /identity/v3/users/cinder => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:46:54.767767 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12792491-0c55-456f-90c9-f30b17b0a692 None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:54.859889 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:54.879763 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12792491-0c55-456f-90c9-f30b17b0a692 None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:54.914478 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca89ead-5ffc-4a96-a70f-6b5a463936a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:54.915087 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca89ead-5ffc-4a96-a70f-6b5a463936a7 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:54.915366 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca89ead-5ffc-4a96-a70f-6b5a463936a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:54.915640 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca89ead-5ffc-4a96-a70f-6b5a463936a7 None admin] PATH_INFO: `/v3/users` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:54.916341 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca89ead-5ffc-4a96-a70f-6b5a463936a7 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81341) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:46:54.916640 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca89ead-5ffc-4a96-a70f-6b5a463936a7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:54.918077 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca89ead-5ffc-4a96-a70f-6b5a463936a7 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:54.937735 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040e70a0-f28e-4683-b6dd-eb5b0b3a869e None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:54.951113 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:54.960665 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 486/981] 10.222.0.33 () {56 vars in 1101 bytes} [Thu Sep 1 14:46:54 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:46:55.062124 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040e70a0-f28e-4683-b6dd-eb5b0b3a869e None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:55.102064 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1349b7b1-ab10-4820-b230-702a36ad608e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:55.103058 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1349b7b1-ab10-4820-b230-702a36ad608e None admin] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:55.103570 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1349b7b1-ab10-4820-b230-702a36ad608e None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:55.104047 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1349b7b1-ab10-4820-b230-702a36ad608e None admin] PATH_INFO: `/v3/services` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:55.104993 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1349b7b1-ab10-4820-b230-702a36ad608e None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:55.106609 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1349b7b1-ab10-4820-b230-702a36ad608e None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:55.127758 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 496/982] 10.222.0.33 () {60 vars in 1141 bytes} [Thu Sep 1 14:46:54 2022] POST /identity/v3/services => generated 242 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 14:46:55.686901 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e04277-0f7a-475b-9138-03cce68627b6 None None] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:55.687386 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e04277-0f7a-475b-9138-03cce68627b6 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:55.687498 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e04277-0f7a-475b-9138-03cce68627b6 None None] PATH_INFO: `/` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:55.688903 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 487/983] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 1 14:46:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:46:55.694485 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e3d134-dab9-4416-87b0-c73b50a1300f None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:55.694775 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e3d134-dab9-4416-87b0-c73b50a1300f None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:55.695311 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e3d134-dab9-4416-87b0-c73b50a1300f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:55.713344 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-83e3d134-dab9-4416-87b0-c73b50a1300f None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:46:55.714876 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-83e3d134-dab9-4416-87b0-c73b50a1300f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:46:55.813506 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:55.838273 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 497/984] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 1 14:46:55 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 14:46:56.099079 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e04277-0f7a-475b-9138-03cce68627b6 None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:56.200853 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e04277-0f7a-475b-9138-03cce68627b6 None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:56.234895 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958d76a5-78af-497e-a99a-617e94e97a95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:56.235521 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958d76a5-78af-497e-a99a-617e94e97a95 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:56.235795 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958d76a5-78af-497e-a99a-617e94e97a95 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:56.236063 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958d76a5-78af-497e-a99a-617e94e97a95 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:56.236639 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958d76a5-78af-497e-a99a-617e94e97a95 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:56.238036 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958d76a5-78af-497e-a99a-617e94e97a95 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:56.246910 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application [None req-958d76a5-78af-497e-a99a-617e94e97a95 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 14:46:56.246910 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:46:56.246910 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 14:46:56.246910 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 14:46:56.246910 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 14:46:56.246910 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 14:46:56.246910 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 14:46:56.246910 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 14:46:56.246910 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 14:46:56.246910 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Sep 01 14:46:56.246910 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 14:46:56.246910 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Sep 01 14:46:56.246910 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:46:56.246910 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:46:56.246910 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:46:56.246910 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:46:56.246910 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:46:56.247961 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:46:56.247961 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:46:56.247961 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:46:56.247961 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:46:56.247961 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:46:56.247961 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:46:56.247961 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 14:46:56.247961 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 14:46:56.247961 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 14:46:56.247961 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 14:46:56.247961 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:46:56.247961 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:46:56.247961 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:46:56.247961 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:46:56.247961 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:46:56.247961 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:46:56.248839 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:46:56.248839 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application with Lock( Sep 01 14:46:56.248839 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:46:56.248839 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:46:56.248839 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:46:56.248839 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:46:56.248839 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:46:56.248839 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:46:56.248839 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:46:56.248839 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:46:56.248839 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 14:46:56.248839 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 14:46:56.248839 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 14:46:56.248839 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Sep 01 14:46:56.252987 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 488/985] 10.222.0.33 () {56 vars in 1126 bytes} [Thu Sep 1 14:46:56 2022] GET /identity/v3/services/object-store => generated 93 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:46:56.254099 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e3d134-dab9-4416-87b0-c73b50a1300f None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:56.355870 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e3d134-dab9-4416-87b0-c73b50a1300f None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:56.388150 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ef6be2-35f2-4a2d-8577-321b0d0fc3c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:56.388749 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ef6be2-35f2-4a2d-8577-321b0d0fc3c7 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:56.389022 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ef6be2-35f2-4a2d-8577-321b0d0fc3c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:56.389284 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ef6be2-35f2-4a2d-8577-321b0d0fc3c7 None admin] PATH_INFO: `/v3/services` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:56.389906 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ef6be2-35f2-4a2d-8577-321b0d0fc3c7 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:46:56.390200 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ef6be2-35f2-4a2d-8577-321b0d0fc3c7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:56.391584 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ef6be2-35f2-4a2d-8577-321b0d0fc3c7 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:56.396983 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 498/986] 10.222.0.33 () {56 vars in 1122 bytes} [Thu Sep 1 14:46:56 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:46:56.402500 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958d76a5-78af-497e-a99a-617e94e97a95 None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:56.405355 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:56.502547 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958d76a5-78af-497e-a99a-617e94e97a95 None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:56.536016 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ef8df3-bf4f-4bfd-ae8a-efc6ba6e9742 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:56.536620 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ef8df3-bf4f-4bfd-ae8a-efc6ba6e9742 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:56.536893 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ef8df3-bf4f-4bfd-ae8a-efc6ba6e9742 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:56.537157 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ef8df3-bf4f-4bfd-ae8a-efc6ba6e9742 None admin] PATH_INFO: `/v3/services` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:56.537775 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ef8df3-bf4f-4bfd-ae8a-efc6ba6e9742 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81341) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:46:56.538069 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ef8df3-bf4f-4bfd-ae8a-efc6ba6e9742 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:56.539472 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ef8df3-bf4f-4bfd-ae8a-efc6ba6e9742 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:56.549371 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 489/987] 10.222.0.33 () {56 vars in 1122 bytes} [Thu Sep 1 14:46:56 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:46:56.550469 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ef6be2-35f2-4a2d-8577-321b0d0fc3c7 None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:56.575439 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc40a429-602d-4207-ae15-6d4bc95ffcd4 None None] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:56.575622 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc40a429-602d-4207-ae15-6d4bc95ffcd4 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:56.575965 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc40a429-602d-4207-ae15-6d4bc95ffcd4 None None] PATH_INFO: `/` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:56.577347 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 490/988] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 1 14:46:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 14:46:56.583115 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5f8179-f827-4314-82ae-8d09de8017e7 None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:56.583405 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5f8179-f827-4314-82ae-8d09de8017e7 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:56.583688 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5f8179-f827-4314-82ae-8d09de8017e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:56.601479 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-5a5f8179-f827-4314-82ae-8d09de8017e7 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:46:56.602763 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-5a5f8179-f827-4314-82ae-8d09de8017e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:46:56.613903 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:56.664684 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ef6be2-35f2-4a2d-8577-321b0d0fc3c7 None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:56.668890 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:56.677429 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 491/989] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 1 14:46:56 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 14:46:56.701034 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7022e62e-4837-482c-9c23-a930da7a0909 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:56.701645 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7022e62e-4837-482c-9c23-a930da7a0909 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:56.701924 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7022e62e-4837-482c-9c23-a930da7a0909 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:56.702191 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7022e62e-4837-482c-9c23-a930da7a0909 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:56.702938 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7022e62e-4837-482c-9c23-a930da7a0909 None admin] RBAC: Adding query filter params (interface=admin, service_id=b17f329e05384e1f9f4c29786d47f5cf, region_id=RegionOne) {{(pid=81342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:46:56.703244 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7022e62e-4837-482c-9c23-a930da7a0909 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:56.704602 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7022e62e-4837-482c-9c23-a930da7a0909 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:56.714939 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 499/990] 10.222.0.33 () {56 vars in 1249 bytes} [Thu Sep 1 14:46:56 2022] GET /identity/v3/endpoints?service_id=b17f329e05384e1f9f4c29786d47f5cf&interface=admin®ion_id=RegionOne => generated 193 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:46:56.715944 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5f8179-f827-4314-82ae-8d09de8017e7 None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:56.816964 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5f8179-f827-4314-82ae-8d09de8017e7 None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:56.850545 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a10e189-fd0c-4c01-839c-eaa0934572d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:56.851152 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a10e189-fd0c-4c01-839c-eaa0934572d9 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:56.851431 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a10e189-fd0c-4c01-839c-eaa0934572d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:56.851699 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a10e189-fd0c-4c01-839c-eaa0934572d9 None admin] PATH_INFO: `/v3/services` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:56.852301 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a10e189-fd0c-4c01-839c-eaa0934572d9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:56.853685 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a10e189-fd0c-4c01-839c-eaa0934572d9 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:56.866571 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 492/991] 10.222.0.33 () {56 vars in 1087 bytes} [Thu Sep 1 14:46:56 2022] GET /identity/v3/services => generated 1727 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:46:56.946222 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7022e62e-4837-482c-9c23-a930da7a0909 None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:57.053767 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7022e62e-4837-482c-9c23-a930da7a0909 None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:57.077204 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:57.087795 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c64adee-d2ae-4eab-ada6-6ed1f2fda016 None None] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:57.088074 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c64adee-d2ae-4eab-ada6-6ed1f2fda016 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:57.088346 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c64adee-d2ae-4eab-ada6-6ed1f2fda016 None None] PATH_INFO: `/` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:57.088808 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a6409c-3a31-4c68-b299-908036c041e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:57.089439 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a6409c-3a31-4c68-b299-908036c041e6 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:57.090304 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a6409c-3a31-4c68-b299-908036c041e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:57.090451 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 493/992] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 1 14:46:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:46:57.090535 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a6409c-3a31-4c68-b299-908036c041e6 None admin] PATH_INFO: `/v3/services/image` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:57.090619 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a6409c-3a31-4c68-b299-908036c041e6 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:57.091940 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a6409c-3a31-4c68-b299-908036c041e6 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:57.094951 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ceeca2-44c5-45c9-929a-1e8ab0685e42 None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:57.095222 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ceeca2-44c5-45c9-929a-1e8ab0685e42 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:57.095493 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ceeca2-44c5-45c9-929a-1e8ab0685e42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:57.101886 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application [None req-46a6409c-3a31-4c68-b299-908036c041e6 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 01 14:46:57.101886 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:46:57.101886 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 14:46:57.101886 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 14:46:57.101886 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 14:46:57.101886 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 14:46:57.101886 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 14:46:57.101886 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 14:46:57.101886 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 01 14:46:57.101886 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Sep 01 14:46:57.101886 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 14:46:57.101886 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Sep 01 14:46:57.101886 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:46:57.101886 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:46:57.101886 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:46:57.101886 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:46:57.101886 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:46:57.102998 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:46:57.102998 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:46:57.102998 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:46:57.102998 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:46:57.102998 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:46:57.102998 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:46:57.102998 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 14:46:57.102998 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 14:46:57.102998 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 14:46:57.102998 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 14:46:57.102998 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:46:57.102998 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:46:57.102998 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:46:57.102998 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:46:57.102998 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:46:57.102998 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:46:57.104428 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:46:57.104428 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application with Lock( Sep 01 14:46:57.104428 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:46:57.104428 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:46:57.104428 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:46:57.104428 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:46:57.104428 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:46:57.104428 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:46:57.104428 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:46:57.104428 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:46:57.104428 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 14:46:57.104428 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 14:46:57.104428 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 01 14:46:57.104428 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Sep 01 14:46:57.104428 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 500/993] 10.222.0.33 () {56 vars in 1105 bytes} [Thu Sep 1 14:46:56 2022] GET /identity/v3/services/image => generated 86 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:46:57.108807 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a6409c-3a31-4c68-b299-908036c041e6 None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:57.113852 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-55ceeca2-44c5-45c9-929a-1e8ab0685e42 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:46:57.115070 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-55ceeca2-44c5-45c9-929a-1e8ab0685e42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:46:57.173475 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:57.186823 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 494/994] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 1 14:46:57 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 14:46:57.216165 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a6409c-3a31-4c68-b299-908036c041e6 None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:57.253011 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d736b921-f213-43b5-865e-a1f4bd653be2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:57.254476 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d736b921-f213-43b5-865e-a1f4bd653be2 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:57.255347 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d736b921-f213-43b5-865e-a1f4bd653be2 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:57.255762 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d736b921-f213-43b5-865e-a1f4bd653be2 None admin] PATH_INFO: `/v3/services` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:57.256533 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d736b921-f213-43b5-865e-a1f4bd653be2 None admin] RBAC: Adding query filter params (name=image) {{(pid=81342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:46:57.256976 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d736b921-f213-43b5-865e-a1f4bd653be2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:57.258516 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d736b921-f213-43b5-865e-a1f4bd653be2 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:57.268025 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 501/995] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 1 14:46:57 2022] GET /identity/v3/services?name=image => generated 122 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 14:46:57.269367 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ceeca2-44c5-45c9-929a-1e8ab0685e42 None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:57.371709 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ceeca2-44c5-45c9-929a-1e8ab0685e42 None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:57.404244 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78491503-673a-453f-80fc-dd4d237fd347 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:57.404847 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78491503-673a-453f-80fc-dd4d237fd347 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:57.405122 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78491503-673a-453f-80fc-dd4d237fd347 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:57.405389 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78491503-673a-453f-80fc-dd4d237fd347 None admin] PATH_INFO: `/v3/services` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:57.406015 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78491503-673a-453f-80fc-dd4d237fd347 None admin] RBAC: Adding query filter params (type=image) {{(pid=81341) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:46:57.406339 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78491503-673a-453f-80fc-dd4d237fd347 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:57.407710 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78491503-673a-453f-80fc-dd4d237fd347 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:57.418274 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 495/996] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 1 14:46:57 2022] GET /identity/v3/services?type=image => generated 122 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:46:57.461570 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d736b921-f213-43b5-865e-a1f4bd653be2 None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:57.520689 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:57.572892 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d736b921-f213-43b5-865e-a1f4bd653be2 None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:57.605937 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba8244a-72e1-481b-86d7-e8a25abc688d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:57.606584 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba8244a-72e1-481b-86d7-e8a25abc688d None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:57.606867 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba8244a-72e1-481b-86d7-e8a25abc688d None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:57.607140 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba8244a-72e1-481b-86d7-e8a25abc688d None admin] PATH_INFO: `/v3/services/compute` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:57.607721 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba8244a-72e1-481b-86d7-e8a25abc688d None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:57.609080 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba8244a-72e1-481b-86d7-e8a25abc688d None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:57.618583 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application [None req-2ba8244a-72e1-481b-86d7-e8a25abc688d None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 01 14:46:57.618583 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:46:57.618583 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 14:46:57.618583 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 14:46:57.618583 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 14:46:57.618583 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 14:46:57.618583 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 14:46:57.618583 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 14:46:57.618583 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 01 14:46:57.618583 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Sep 01 14:46:57.618583 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 14:46:57.618583 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Sep 01 14:46:57.618583 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:46:57.618583 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:46:57.618583 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:46:57.618583 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:46:57.618583 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:46:57.619778 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:46:57.619778 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:46:57.619778 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:46:57.619778 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:46:57.619778 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:46:57.619778 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:46:57.619778 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 14:46:57.619778 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 14:46:57.619778 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 14:46:57.619778 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 14:46:57.619778 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:46:57.619778 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:46:57.619778 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:46:57.619778 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:46:57.619778 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:46:57.619778 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:46:57.620674 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:46:57.620674 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application with Lock( Sep 01 14:46:57.620674 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:46:57.620674 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:46:57.620674 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:46:57.620674 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:46:57.620674 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:46:57.620674 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:46:57.620674 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:46:57.620674 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:46:57.620674 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 14:46:57.620674 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 14:46:57.620674 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 01 14:46:57.620674 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Sep 01 14:46:57.620674 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 502/997] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Sep 1 14:46:57 2022] GET /identity/v3/services/compute => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:46:57.625522 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78491503-673a-453f-80fc-dd4d237fd347 None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:57.686379 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:57.733165 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78491503-673a-453f-80fc-dd4d237fd347 None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:57.766458 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2073aa1f-d733-4fc2-a927-7d659d902b43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:57.767061 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2073aa1f-d733-4fc2-a927-7d659d902b43 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:57.767355 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2073aa1f-d733-4fc2-a927-7d659d902b43 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:57.767613 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2073aa1f-d733-4fc2-a927-7d659d902b43 None admin] PATH_INFO: `/v3/services` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:57.768229 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2073aa1f-d733-4fc2-a927-7d659d902b43 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81341) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:46:57.768527 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2073aa1f-d733-4fc2-a927-7d659d902b43 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:57.770442 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2073aa1f-d733-4fc2-a927-7d659d902b43 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:57.779717 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 496/998] 10.222.0.33 () {56 vars in 1112 bytes} [Thu Sep 1 14:46:57 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:46:57.781226 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba8244a-72e1-481b-86d7-e8a25abc688d None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:57.885329 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba8244a-72e1-481b-86d7-e8a25abc688d None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:57.902441 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:57.922265 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27182fc-2f3b-425d-ba5d-3df7060666c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:57.922878 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27182fc-2f3b-425d-ba5d-3df7060666c6 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:57.923161 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27182fc-2f3b-425d-ba5d-3df7060666c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:57.923434 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27182fc-2f3b-425d-ba5d-3df7060666c6 None admin] PATH_INFO: `/v3/services` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:57.924073 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27182fc-2f3b-425d-ba5d-3df7060666c6 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:46:57.924378 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27182fc-2f3b-425d-ba5d-3df7060666c6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:57.925782 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27182fc-2f3b-425d-ba5d-3df7060666c6 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:57.935541 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 503/999] 10.222.0.33 () {56 vars in 1112 bytes} [Thu Sep 1 14:46:57 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:46:57.936880 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2073aa1f-d733-4fc2-a927-7d659d902b43 None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:58.039833 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2073aa1f-d733-4fc2-a927-7d659d902b43 None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:58.056886 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:58.075341 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9a1041-990e-4bb3-aafe-b3784015a610 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:58.075943 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9a1041-990e-4bb3-aafe-b3784015a610 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:58.076223 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9a1041-990e-4bb3-aafe-b3784015a610 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:58.076494 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9a1041-990e-4bb3-aafe-b3784015a610 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:58.077187 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9a1041-990e-4bb3-aafe-b3784015a610 None admin] RBAC: Adding query filter params (interface=public, service_id=84f75d9ebc74470daae3e4116fecf23f, region_id=RegionOne) {{(pid=81341) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:46:58.077490 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9a1041-990e-4bb3-aafe-b3784015a610 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:58.078914 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9a1041-990e-4bb3-aafe-b3784015a610 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:58.088841 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 497/1000] 10.222.0.33 () {56 vars in 1251 bytes} [Thu Sep 1 14:46:57 2022] GET /identity/v3/endpoints?service_id=84f75d9ebc74470daae3e4116fecf23f&interface=public®ion_id=RegionOne => generated 194 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:46:58.090395 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27182fc-2f3b-425d-ba5d-3df7060666c6 None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:58.189455 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27182fc-2f3b-425d-ba5d-3df7060666c6 None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:58.223530 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d989ab5-e2e8-44f9-b252-8ed7ff7240fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:58.224128 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d989ab5-e2e8-44f9-b252-8ed7ff7240fe None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:58.224404 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d989ab5-e2e8-44f9-b252-8ed7ff7240fe None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:58.224664 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d989ab5-e2e8-44f9-b252-8ed7ff7240fe None admin] PATH_INFO: `/v3/services` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:58.225254 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d989ab5-e2e8-44f9-b252-8ed7ff7240fe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:58.226650 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d989ab5-e2e8-44f9-b252-8ed7ff7240fe None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:58.232933 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 504/1001] 10.222.0.33 () {56 vars in 1087 bytes} [Thu Sep 1 14:46:58 2022] GET /identity/v3/services => generated 1727 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:46:58.776080 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b62e768-ad2b-4609-9114-93c8c4dc5668 None None] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:58.776594 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b62e768-ad2b-4609-9114-93c8c4dc5668 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:58.776718 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b62e768-ad2b-4609-9114-93c8c4dc5668 None None] PATH_INFO: `/` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:58.777981 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 498/1002] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 1 14:46:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:46:58.782951 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df29652-e48c-4c84-b3a4-63d8ad100919 None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:58.783343 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df29652-e48c-4c84-b3a4-63d8ad100919 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:58.783504 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df29652-e48c-4c84-b3a4-63d8ad100919 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:58.801894 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-5df29652-e48c-4c84-b3a4-63d8ad100919 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:46:58.803087 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-5df29652-e48c-4c84-b3a4-63d8ad100919 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:46:58.842861 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:58.871496 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 505/1003] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 1 14:46:58 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 14:46:59.137150 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b62e768-ad2b-4609-9114-93c8c4dc5668 None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:59.241733 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b62e768-ad2b-4609-9114-93c8c4dc5668 None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:59.275282 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868e5102-9fd4-4ed3-851f-45922d20c967 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:59.275908 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868e5102-9fd4-4ed3-851f-45922d20c967 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:59.276184 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868e5102-9fd4-4ed3-851f-45922d20c967 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:59.276454 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868e5102-9fd4-4ed3-851f-45922d20c967 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:59.277022 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868e5102-9fd4-4ed3-851f-45922d20c967 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:59.278421 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868e5102-9fd4-4ed3-851f-45922d20c967 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:59.287566 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application [None req-868e5102-9fd4-4ed3-851f-45922d20c967 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 14:46:59.287566 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:46:59.287566 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 14:46:59.287566 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 14:46:59.287566 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 14:46:59.287566 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 14:46:59.287566 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 14:46:59.287566 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 14:46:59.287566 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 14:46:59.287566 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Sep 01 14:46:59.287566 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 14:46:59.287566 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Sep 01 14:46:59.287566 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:46:59.287566 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:46:59.287566 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:46:59.287566 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:46:59.287566 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:46:59.288570 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:46:59.288570 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:46:59.288570 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:46:59.288570 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:46:59.288570 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:46:59.288570 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:46:59.288570 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 14:46:59.288570 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 14:46:59.288570 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 14:46:59.288570 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 14:46:59.288570 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:46:59.288570 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:46:59.288570 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:46:59.288570 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:46:59.288570 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:46:59.288570 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:46:59.289464 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:46:59.289464 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application with Lock( Sep 01 14:46:59.289464 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:46:59.289464 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:46:59.289464 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:46:59.289464 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:46:59.289464 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:46:59.289464 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:46:59.289464 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:46:59.289464 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:46:59.289464 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 14:46:59.289464 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 14:46:59.289464 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 14:46:59.289464 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Sep 01 14:46:59.289464 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 499/1004] 10.222.0.33 () {56 vars in 1126 bytes} [Thu Sep 1 14:46:59 2022] GET /identity/v3/services/object-store => generated 93 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:46:59.294523 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df29652-e48c-4c84-b3a4-63d8ad100919 None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:59.351720 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb74d949-3300-4c93-821a-f2cd358d84a5 None None] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:59.352041 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb74d949-3300-4c93-821a-f2cd358d84a5 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:59.352274 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb74d949-3300-4c93-821a-f2cd358d84a5 None None] PATH_INFO: `/` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:59.353753 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 500/1005] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 1 14:46:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:46:59.358341 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f10624-b02e-460f-9429-7ca0ecece46c None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:59.358629 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f10624-b02e-460f-9429-7ca0ecece46c None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:59.358905 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f10624-b02e-460f-9429-7ca0ecece46c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:59.376850 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-48f10624-b02e-460f-9429-7ca0ecece46c None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:46:59.378053 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-48f10624-b02e-460f-9429-7ca0ecece46c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:46:59.405424 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df29652-e48c-4c84-b3a4-63d8ad100919 None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:59.421578 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:59.441285 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78640394-851c-4c0b-8895-d6482f8bccb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:59.441894 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78640394-851c-4c0b-8895-d6482f8bccb3 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:59.442171 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78640394-851c-4c0b-8895-d6482f8bccb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:59.442466 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78640394-851c-4c0b-8895-d6482f8bccb3 None admin] PATH_INFO: `/v3/services` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:59.443136 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78640394-851c-4c0b-8895-d6482f8bccb3 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:46:59.443388 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78640394-851c-4c0b-8895-d6482f8bccb3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:59.444753 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78640394-851c-4c0b-8895-d6482f8bccb3 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:59.449391 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 501/1006] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 1 14:46:59 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 14:46:59.449852 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 506/1007] 10.222.0.33 () {56 vars in 1122 bytes} [Thu Sep 1 14:46:59 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:46:59.455291 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f10624-b02e-460f-9429-7ca0ecece46c None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:59.553808 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f10624-b02e-460f-9429-7ca0ecece46c None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:59.589106 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ffda95-3278-4644-b73a-f1a0fe10dd4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:59.589708 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ffda95-3278-4644-b73a-f1a0fe10dd4e None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:59.589989 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ffda95-3278-4644-b73a-f1a0fe10dd4e None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:59.590281 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ffda95-3278-4644-b73a-f1a0fe10dd4e None admin] PATH_INFO: `/v3/services` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:59.590909 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ffda95-3278-4644-b73a-f1a0fe10dd4e None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81341) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:46:59.591208 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ffda95-3278-4644-b73a-f1a0fe10dd4e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:59.592601 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ffda95-3278-4644-b73a-f1a0fe10dd4e None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:59.602036 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 502/1008] 10.222.0.33 () {56 vars in 1122 bytes} [Thu Sep 1 14:46:59 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 14:46:59.603084 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78640394-851c-4c0b-8895-d6482f8bccb3 None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:59.644757 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:59.712451 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78640394-851c-4c0b-8895-d6482f8bccb3 None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:59.724756 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ffda95-3278-4644-b73a-f1a0fe10dd4e None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:46:59.749149 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff30f69a-dd6a-4143-aba6-d9041c2db21a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:59.749775 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff30f69a-dd6a-4143-aba6-d9041c2db21a None admin] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:59.750053 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff30f69a-dd6a-4143-aba6-d9041c2db21a None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:59.750428 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff30f69a-dd6a-4143-aba6-d9041c2db21a None admin] PATH_INFO: `/v3/endpoints` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:59.751092 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff30f69a-dd6a-4143-aba6-d9041c2db21a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:59.752490 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff30f69a-dd6a-4143-aba6-d9041c2db21a None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:59.770030 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:46:59.771802 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 507/1009] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Sep 1 14:46:59 2022] POST /identity/v3/endpoints => generated 325 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 14:46:59.835597 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ffda95-3278-4644-b73a-f1a0fe10dd4e None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:46:59.878029 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07dd2de-31b9-455a-9b8c-fed8a0507c02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:46:59.879211 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07dd2de-31b9-455a-9b8c-fed8a0507c02 None admin] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:46:59.879488 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07dd2de-31b9-455a-9b8c-fed8a0507c02 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:46:59.879752 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07dd2de-31b9-455a-9b8c-fed8a0507c02 None admin] PATH_INFO: `/v3/services` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:46:59.880421 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07dd2de-31b9-455a-9b8c-fed8a0507c02 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:46:59.881834 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07dd2de-31b9-455a-9b8c-fed8a0507c02 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:46:59.900396 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 503/1010] 10.222.0.33 () {60 vars in 1141 bytes} [Thu Sep 1 14:46:59 2022] POST /identity/v3/services => generated 242 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 14:47:00.150138 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7218873-c561-4782-a6b6-2cbac3c7e968 None None] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:47:00.150684 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7218873-c561-4782-a6b6-2cbac3c7e968 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:47:00.150684 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7218873-c561-4782-a6b6-2cbac3c7e968 None None] PATH_INFO: `/` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:47:00.152165 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 508/1011] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 1 14:47:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:47:00.157198 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530bd600-b14a-4712-bd56-f5199ad06557 None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:47:00.157481 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530bd600-b14a-4712-bd56-f5199ad06557 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:47:00.157749 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530bd600-b14a-4712-bd56-f5199ad06557 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:47:00.176149 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-530bd600-b14a-4712-bd56-f5199ad06557 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:47:00.177333 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-530bd600-b14a-4712-bd56-f5199ad06557 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:47:00.251746 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 504/1012] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 1 14:47:00 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 14:47:00.517844 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7218873-c561-4782-a6b6-2cbac3c7e968 None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:47:00.615404 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:47:00.620642 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7218873-c561-4782-a6b6-2cbac3c7e968 None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:47:00.658503 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a587a4d8-a8d7-4eb9-9466-9a1da0e02951 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:47:00.659137 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a587a4d8-a8d7-4eb9-9466-9a1da0e02951 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:47:00.659416 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a587a4d8-a8d7-4eb9-9466-9a1da0e02951 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:47:00.660174 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a587a4d8-a8d7-4eb9-9466-9a1da0e02951 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:47:00.660759 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a587a4d8-a8d7-4eb9-9466-9a1da0e02951 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:47:00.662178 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a587a4d8-a8d7-4eb9-9466-9a1da0e02951 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:47:00.671345 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application [None req-a587a4d8-a8d7-4eb9-9466-9a1da0e02951 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 01 14:47:00.671345 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:47:00.671345 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 14:47:00.671345 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 14:47:00.671345 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 14:47:00.671345 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 14:47:00.671345 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 14:47:00.671345 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 14:47:00.671345 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 01 14:47:00.671345 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Sep 01 14:47:00.671345 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 14:47:00.671345 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Sep 01 14:47:00.671345 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:47:00.671345 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:47:00.671345 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:47:00.671345 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:47:00.671345 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:47:00.672346 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:47:00.672346 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:47:00.672346 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:47:00.672346 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:47:00.672346 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:47:00.672346 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:47:00.672346 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 14:47:00.672346 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 14:47:00.672346 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 14:47:00.672346 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 14:47:00.672346 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:47:00.672346 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:47:00.672346 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:47:00.672346 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:47:00.672346 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:47:00.672346 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:47:00.674963 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:47:00.674963 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application with Lock( Sep 01 14:47:00.674963 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:47:00.674963 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:47:00.674963 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:47:00.674963 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:47:00.674963 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:47:00.674963 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:47:00.674963 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:47:00.674963 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:47:00.674963 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 14:47:00.674963 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 14:47:00.674963 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 01 14:47:00.674963 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Sep 01 14:47:00.674963 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 509/1013] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Sep 1 14:47:00 2022] GET /identity/v3/services/compute => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:47:00.679650 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530bd600-b14a-4712-bd56-f5199ad06557 None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:47:00.703387 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:47:00.782343 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530bd600-b14a-4712-bd56-f5199ad06557 None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:47:00.815621 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d57b48-4705-4753-bc83-ad4f0028f657 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:47:00.816251 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d57b48-4705-4753-bc83-ad4f0028f657 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:47:00.816526 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d57b48-4705-4753-bc83-ad4f0028f657 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:47:00.816793 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d57b48-4705-4753-bc83-ad4f0028f657 None admin] PATH_INFO: `/v3/services` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:47:00.817415 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d57b48-4705-4753-bc83-ad4f0028f657 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81341) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:47:00.817710 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d57b48-4705-4753-bc83-ad4f0028f657 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:47:00.819114 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d57b48-4705-4753-bc83-ad4f0028f657 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:47:00.828775 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 505/1014] 10.222.0.33 () {56 vars in 1112 bytes} [Thu Sep 1 14:47:00 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:47:00.830042 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a587a4d8-a8d7-4eb9-9466-9a1da0e02951 None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:47:00.936297 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a587a4d8-a8d7-4eb9-9466-9a1da0e02951 None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:47:00.969504 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dba66d0-a073-4cb7-a1e3-3335f53b9291 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:47:00.970119 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dba66d0-a073-4cb7-a1e3-3335f53b9291 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:47:00.970426 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dba66d0-a073-4cb7-a1e3-3335f53b9291 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:47:00.970695 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dba66d0-a073-4cb7-a1e3-3335f53b9291 None admin] PATH_INFO: `/v3/services` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:47:00.971337 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dba66d0-a073-4cb7-a1e3-3335f53b9291 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:47:00.971634 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dba66d0-a073-4cb7-a1e3-3335f53b9291 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:47:00.973093 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dba66d0-a073-4cb7-a1e3-3335f53b9291 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:47:00.978852 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 510/1015] 10.222.0.33 () {56 vars in 1112 bytes} [Thu Sep 1 14:47:00 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:47:00.984439 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d57b48-4705-4753-bc83-ad4f0028f657 None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:47:01.069458 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:47:01.085427 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d57b48-4705-4753-bc83-ad4f0028f657 None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:47:01.118774 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90502e25-2bd5-420a-b145-a684d8e05911 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:47:01.119397 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90502e25-2bd5-420a-b145-a684d8e05911 None admin] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:47:01.119674 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90502e25-2bd5-420a-b145-a684d8e05911 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:47:01.119940 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90502e25-2bd5-420a-b145-a684d8e05911 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:47:01.120607 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90502e25-2bd5-420a-b145-a684d8e05911 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:47:01.121966 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90502e25-2bd5-420a-b145-a684d8e05911 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:47:01.148417 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 506/1016] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Sep 1 14:47:00 2022] POST /identity/v3/endpoints => generated 334 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:47:01.723638 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926fb52c-981c-4b90-adad-62514b4d37c3 None None] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:47:01.724147 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926fb52c-981c-4b90-adad-62514b4d37c3 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:47:01.724225 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926fb52c-981c-4b90-adad-62514b4d37c3 None None] PATH_INFO: `/` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:47:01.725653 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 511/1017] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 1 14:47:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:47:01.730648 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89720480-6721-4bec-9ef4-9bd2283625b3 None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:47:01.730940 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89720480-6721-4bec-9ef4-9bd2283625b3 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:47:01.731218 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89720480-6721-4bec-9ef4-9bd2283625b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:47:01.749183 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-89720480-6721-4bec-9ef4-9bd2283625b3 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:47:01.750414 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-89720480-6721-4bec-9ef4-9bd2283625b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:47:01.809789 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8003a70e-8b55-4f49-ac59-395c3760850a None None] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:47:01.810057 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8003a70e-8b55-4f49-ac59-395c3760850a None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:47:01.810351 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8003a70e-8b55-4f49-ac59-395c3760850a None None] PATH_INFO: `/` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:47:01.811677 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 512/1018] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 1 14:47:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 14:47:01.817219 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba4abc8-7fec-4ab4-9b7e-d836d4754c23 None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:47:01.817734 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba4abc8-7fec-4ab4-9b7e-d836d4754c23 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:47:01.818211 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba4abc8-7fec-4ab4-9b7e-d836d4754c23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:47:01.830278 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 507/1019] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 1 14:47:01 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 14:47:01.841263 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-eba4abc8-7fec-4ab4-9b7e-d836d4754c23 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:47:01.842468 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-eba4abc8-7fec-4ab4-9b7e-d836d4754c23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:47:01.862774 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:47:01.910479 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 513/1020] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 1 14:47:01 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 14:47:02.095304 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89720480-6721-4bec-9ef4-9bd2283625b3 None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:47:02.160722 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:47:02.176541 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba4abc8-7fec-4ab4-9b7e-d836d4754c23 None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:47:02.201037 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89720480-6721-4bec-9ef4-9bd2283625b3 None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:47:02.247712 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb50aef5-49bc-4c84-afae-acccd4fa4a2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:47:02.248587 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb50aef5-49bc-4c84-afae-acccd4fa4a2b None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:47:02.249715 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb50aef5-49bc-4c84-afae-acccd4fa4a2b None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:47:02.249991 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb50aef5-49bc-4c84-afae-acccd4fa4a2b None admin] PATH_INFO: `/v3/domains/default` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:47:02.251126 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb50aef5-49bc-4c84-afae-acccd4fa4a2b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:47:02.252566 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb50aef5-49bc-4c84-afae-acccd4fa4a2b None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:47:02.254885 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 508/1021] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 1 14:47:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:47:02.260574 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb50aef5-49bc-4c84-afae-acccd4fa4a2b None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:47:02.284544 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba4abc8-7fec-4ab4-9b7e-d836d4754c23 None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:47:02.322507 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60b5688-c80d-4d0b-94d9-1cd9f29e16fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:47:02.323157 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60b5688-c80d-4d0b-94d9-1cd9f29e16fa None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:47:02.323434 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60b5688-c80d-4d0b-94d9-1cd9f29e16fa None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:47:02.323704 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60b5688-c80d-4d0b-94d9-1cd9f29e16fa None admin] PATH_INFO: `/v3/services/image` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:47:02.324282 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60b5688-c80d-4d0b-94d9-1cd9f29e16fa None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:47:02.325660 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60b5688-c80d-4d0b-94d9-1cd9f29e16fa None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:47:02.334479 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application [None req-f60b5688-c80d-4d0b-94d9-1cd9f29e16fa None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 01 14:47:02.334479 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:47:02.334479 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 14:47:02.334479 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 14:47:02.334479 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 14:47:02.334479 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 14:47:02.334479 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 14:47:02.334479 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 14:47:02.334479 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 01 14:47:02.334479 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Sep 01 14:47:02.334479 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 14:47:02.334479 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Sep 01 14:47:02.334479 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:47:02.334479 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:47:02.334479 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:47:02.334479 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:47:02.334479 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:47:02.335478 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:47:02.335478 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:47:02.335478 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:47:02.335478 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:47:02.335478 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:47:02.335478 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:47:02.335478 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 14:47:02.335478 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 14:47:02.335478 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 14:47:02.335478 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 14:47:02.335478 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:47:02.335478 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:47:02.335478 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:47:02.335478 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:47:02.335478 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:47:02.335478 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:47:02.337113 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:47:02.337113 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application with Lock( Sep 01 14:47:02.337113 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:47:02.337113 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:47:02.337113 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:47:02.337113 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:47:02.337113 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:47:02.337113 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:47:02.337113 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:47:02.337113 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:47:02.337113 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 14:47:02.337113 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 14:47:02.337113 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 01 14:47:02.337113 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Sep 01 14:47:02.337113 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 514/1022] 10.222.0.33 () {56 vars in 1105 bytes} [Thu Sep 1 14:47:02 2022] GET /identity/v3/services/image => generated 86 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 14:47:02.341320 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60b5688-c80d-4d0b-94d9-1cd9f29e16fa None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:47:02.366161 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:47:02.369419 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb50aef5-49bc-4c84-afae-acccd4fa4a2b None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:47:02.403999 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bef85c-2590-432e-8126-c3e244c4d8d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:47:02.404601 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bef85c-2590-432e-8126-c3e244c4d8d6 None admin] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:47:02.404883 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bef85c-2590-432e-8126-c3e244c4d8d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:47:02.405155 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bef85c-2590-432e-8126-c3e244c4d8d6 None admin] PATH_INFO: `/v3/projects` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:47:02.405791 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2bef85c-2590-432e-8126-c3e244c4d8d6 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:47:02.407260 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2bef85c-2590-432e-8126-c3e244c4d8d6 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:47:02.412512 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:47:02.437060 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 509/1023] 10.222.0.33 () {60 vars in 1141 bytes} [Thu Sep 1 14:47:02 2022] POST /identity/v3/projects => generated 311 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:47:02.464922 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60b5688-c80d-4d0b-94d9-1cd9f29e16fa None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:47:02.502408 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550add70-0e71-4bf1-907d-1f8b73f305ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:47:02.503027 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550add70-0e71-4bf1-907d-1f8b73f305ef None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:47:02.503365 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550add70-0e71-4bf1-907d-1f8b73f305ef None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:47:02.504137 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550add70-0e71-4bf1-907d-1f8b73f305ef None admin] PATH_INFO: `/v3/services` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:47:02.504774 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550add70-0e71-4bf1-907d-1f8b73f305ef None admin] RBAC: Adding query filter params (name=image) {{(pid=81342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:47:02.505067 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550add70-0e71-4bf1-907d-1f8b73f305ef None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:47:02.506479 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550add70-0e71-4bf1-907d-1f8b73f305ef None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:47:02.512107 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 515/1024] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 1 14:47:02 2022] GET /identity/v3/services?name=image => generated 122 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 14:47:02.517852 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bef85c-2590-432e-8126-c3e244c4d8d6 None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:47:02.628828 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bef85c-2590-432e-8126-c3e244c4d8d6 None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:47:02.671960 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2e5638-eb35-4f4d-af8b-f1699c05de09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:47:02.672572 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2e5638-eb35-4f4d-af8b-f1699c05de09 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:47:02.672846 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2e5638-eb35-4f4d-af8b-f1699c05de09 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:47:02.673111 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2e5638-eb35-4f4d-af8b-f1699c05de09 None admin] PATH_INFO: `/v3/services` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:47:02.673742 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2e5638-eb35-4f4d-af8b-f1699c05de09 None admin] RBAC: Adding query filter params (type=image) {{(pid=81341) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:47:02.674041 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2e5638-eb35-4f4d-af8b-f1699c05de09 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:47:02.676062 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2e5638-eb35-4f4d-af8b-f1699c05de09 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:47:02.681206 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 510/1025] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 1 14:47:02 2022] GET /identity/v3/services?type=image => generated 350 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:47:02.686856 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550add70-0e71-4bf1-907d-1f8b73f305ef None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:47:02.772639 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:47:02.791005 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550add70-0e71-4bf1-907d-1f8b73f305ef None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:47:02.824534 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88a7cfb-df19-41e8-acc9-22f0ca3a1734 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:47:02.825137 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88a7cfb-df19-41e8-acc9-22f0ca3a1734 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:47:02.825409 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88a7cfb-df19-41e8-acc9-22f0ca3a1734 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:47:02.825673 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88a7cfb-df19-41e8-acc9-22f0ca3a1734 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:47:02.826396 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88a7cfb-df19-41e8-acc9-22f0ca3a1734 None admin] RBAC: Adding query filter params (interface=public, service_id=0c4aa8af2fcf43c4ac816551043cf998, region_id=RegionOne) {{(pid=81342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:47:02.826700 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88a7cfb-df19-41e8-acc9-22f0ca3a1734 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:47:02.828055 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88a7cfb-df19-41e8-acc9-22f0ca3a1734 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:47:02.833279 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 516/1026] 10.222.0.33 () {56 vars in 1251 bytes} [Thu Sep 1 14:47:02 2022] GET /identity/v3/endpoints?service_id=0c4aa8af2fcf43c4ac816551043cf998&interface=public®ion_id=RegionOne => generated 194 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 14:47:02.839007 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2e5638-eb35-4f4d-af8b-f1699c05de09 None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:47:02.893175 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:47:02.942613 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2e5638-eb35-4f4d-af8b-f1699c05de09 None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:47:02.978334 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc91048-b8db-4634-9241-892e0357ce3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:47:02.979299 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc91048-b8db-4634-9241-892e0357ce3b None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:47:02.979779 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc91048-b8db-4634-9241-892e0357ce3b None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:47:02.980249 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc91048-b8db-4634-9241-892e0357ce3b None admin] PATH_INFO: `/v3/services` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:47:02.981064 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc91048-b8db-4634-9241-892e0357ce3b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:47:02.982677 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc91048-b8db-4634-9241-892e0357ce3b None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:47:02.989698 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 511/1027] 10.222.0.33 () {56 vars in 1087 bytes} [Thu Sep 1 14:47:02 2022] GET /identity/v3/services => generated 1957 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 14:47:03.131044 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097fb990-11df-4682-a0da-0c8591782cdd None None] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:47:03.131329 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097fb990-11df-4682-a0da-0c8591782cdd None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:47:03.131602 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097fb990-11df-4682-a0da-0c8591782cdd None None] PATH_INFO: `/` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:47:03.132942 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 517/1028] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 1 14:47:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:47:03.137998 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3a0693-5f63-4f96-b1d5-f600e07454f0 None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:47:03.138455 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3a0693-5f63-4f96-b1d5-f600e07454f0 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:47:03.138657 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3a0693-5f63-4f96-b1d5-f600e07454f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:47:03.159360 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-9b3a0693-5f63-4f96-b1d5-f600e07454f0 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:47:03.161123 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-9b3a0693-5f63-4f96-b1d5-f600e07454f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:47:03.230284 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 512/1029] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 1 14:47:03 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 14:47:03.511794 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097fb990-11df-4682-a0da-0c8591782cdd None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:47:03.615820 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097fb990-11df-4682-a0da-0c8591782cdd None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:47:03.650082 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2831258-da80-44b9-a28c-225360ae3f2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:47:03.650750 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2831258-da80-44b9-a28c-225360ae3f2d None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:47:03.651025 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2831258-da80-44b9-a28c-225360ae3f2d None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:47:03.651292 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2831258-da80-44b9-a28c-225360ae3f2d None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:47:03.663804 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2831258-da80-44b9-a28c-225360ae3f2d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 14:47:03.668650 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application [None req-a2831258-da80-44b9-a28c-225360ae3f2d None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 01 14:47:03.668650 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:47:03.668650 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:47:03.668650 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:47:03.668650 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:47:03.668650 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:47:03.668650 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:47:03.668650 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:47:03.668650 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:47:03.668650 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:47:03.668650 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:47:03.668650 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:47:03.668650 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 14:47:03.668650 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 14:47:03.668650 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 14:47:03.668650 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 14:47:03.669569 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 14:47:03.669569 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 14:47:03.669569 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 14:47:03.669569 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 14:47:03.669569 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:47:03.669569 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:47:03.669569 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:47:03.669569 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:47:03.669569 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:47:03.669569 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:47:03.669569 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:47:03.669569 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application with Lock( Sep 01 14:47:03.669569 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:47:03.669569 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:47:03.669569 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:47:03.669569 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:47:03.670461 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:47:03.670461 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:47:03.670461 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:47:03.670461 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:47:03.670461 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 14:47:03.670461 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 14:47:03.670461 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 14:47:03.670461 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 14:47:03.670461 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 14:47:03.670461 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 14:47:03.670461 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 01 14:47:03.670461 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Sep 01 14:47:03.670461 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 518/1030] 10.222.0.33 () {56 vars in 1120 bytes} [Thu Sep 1 14:47:03 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:47:03.675614 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3a0693-5f63-4f96-b1d5-f600e07454f0 None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:47:03.723467 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:47:03.777666 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3a0693-5f63-4f96-b1d5-f600e07454f0 None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:47:03.810710 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420f3838-7cd4-4730-8555-978fc66d1912 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:47:03.811318 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420f3838-7cd4-4730-8555-978fc66d1912 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:47:03.811595 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420f3838-7cd4-4730-8555-978fc66d1912 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:47:03.811864 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420f3838-7cd4-4730-8555-978fc66d1912 None admin] PATH_INFO: `/v3/roles` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:47:03.812527 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420f3838-7cd4-4730-8555-978fc66d1912 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81341) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:47:03.812826 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420f3838-7cd4-4730-8555-978fc66d1912 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:47:03.814203 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420f3838-7cd4-4730-8555-978fc66d1912 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:47:03.827570 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 513/1031] 10.222.0.33 () {56 vars in 1115 bytes} [Thu Sep 1 14:47:03 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 14:47:03.828851 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2831258-da80-44b9-a28c-225360ae3f2d None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:47:03.841675 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:47:03.936579 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2831258-da80-44b9-a28c-225360ae3f2d None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:47:03.972643 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904622b5-2620-4325-849c-858a67580629 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:47:03.973275 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904622b5-2620-4325-849c-858a67580629 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:47:03.973552 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904622b5-2620-4325-849c-858a67580629 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:47:03.973822 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904622b5-2620-4325-849c-858a67580629 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:47:03.984494 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904622b5-2620-4325-849c-858a67580629 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:47:03.985962 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904622b5-2620-4325-849c-858a67580629 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:47:04.001215 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application [None req-904622b5-2620-4325-849c-858a67580629 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 14:47:04.001215 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:47:04.001215 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:47:04.001215 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:47:04.001215 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:47:04.001215 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:47:04.001215 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:47:04.001215 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:47:04.001215 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:47:04.001215 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:47:04.001215 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:47:04.001215 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:47:04.001215 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 14:47:04.001215 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 14:47:04.001215 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 14:47:04.001215 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 14:47:04.002230 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:47:04.002230 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:47:04.002230 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:47:04.002230 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:47:04.002230 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:47:04.002230 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:47:04.002230 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:47:04.002230 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application with Lock( Sep 01 14:47:04.002230 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:47:04.002230 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:47:04.002230 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:47:04.002230 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:47:04.002230 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:47:04.002230 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:47:04.002230 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:47:04.002230 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:47:04.002230 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 14:47:04.004346 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 14:47:04.004346 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 14:47:04.004346 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Sep 01 14:47:04.004779 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 519/1032] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 1 14:47:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:47:04.009126 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420f3838-7cd4-4730-8555-978fc66d1912 None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:47:04.116730 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420f3838-7cd4-4730-8555-978fc66d1912 None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:47:04.125863 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:47:04.151460 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8bf919-db99-4bc7-ad9a-c85173c2be9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:47:04.152105 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8bf919-db99-4bc7-ad9a-c85173c2be9d None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:47:04.152326 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8bf919-db99-4bc7-ad9a-c85173c2be9d None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:47:04.152590 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8bf919-db99-4bc7-ad9a-c85173c2be9d None admin] PATH_INFO: `/v3/domains` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:47:04.153219 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8bf919-db99-4bc7-ad9a-c85173c2be9d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81341) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:47:04.153515 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8bf919-db99-4bc7-ad9a-c85173c2be9d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:47:04.154910 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8bf919-db99-4bc7-ad9a-c85173c2be9d None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:47:04.168145 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 514/1033] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Sep 1 14:47:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:47:04.173615 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904622b5-2620-4325-849c-858a67580629 None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:47:04.232462 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:47:04.274104 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904622b5-2620-4325-849c-858a67580629 None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:47:04.307632 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae1f595-48ab-4221-bd9b-920a74956d5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:47:04.308254 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae1f595-48ab-4221-bd9b-920a74956d5d None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:47:04.308527 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae1f595-48ab-4221-bd9b-920a74956d5d None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:47:04.308792 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae1f595-48ab-4221-bd9b-920a74956d5d None admin] PATH_INFO: `/v3/users/nova` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:47:04.316338 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae1f595-48ab-4221-bd9b-920a74956d5d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:47:04.317759 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae1f595-48ab-4221-bd9b-920a74956d5d None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:47:04.328876 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application [None req-eae1f595-48ab-4221-bd9b-920a74956d5d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 01 14:47:04.328876 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:47:04.328876 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:47:04.328876 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:47:04.328876 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:47:04.328876 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:47:04.328876 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:47:04.328876 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:47:04.328876 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:47:04.328876 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:47:04.328876 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:47:04.328876 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:47:04.328876 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 14:47:04.328876 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 14:47:04.328876 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 14:47:04.328876 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 14:47:04.329858 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:47:04.329858 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:47:04.329858 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 14:47:04.329858 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 14:47:04.329858 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 14:47:04.329858 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 14:47:04.329858 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:47:04.329858 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:47:04.329858 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:47:04.329858 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:47:04.329858 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:47:04.329858 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application with Lock( Sep 01 14:47:04.329858 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:47:04.329858 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:47:04.329858 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:47:04.329858 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:47:04.329858 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:47:04.329858 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:47:04.331243 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:47:04.331243 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:47:04.331243 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 14:47:04.331243 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 14:47:04.331243 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 14:47:04.331243 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 14:47:04.331243 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 14:47:04.331243 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 14:47:04.331243 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 01 14:47:04.331243 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Sep 01 14:47:04.334595 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 520/1034] 10.222.0.33 () {56 vars in 1093 bytes} [Thu Sep 1 14:47:04 2022] GET /identity/v3/users/nova => generated 82 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:47:04.335811 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8bf919-db99-4bc7-ad9a-c85173c2be9d None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:47:04.393857 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d0b542-bbd1-4c08-838d-5e0ee96bb632 None None] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:47:04.394075 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d0b542-bbd1-4c08-838d-5e0ee96bb632 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:47:04.394374 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d0b542-bbd1-4c08-838d-5e0ee96bb632 None None] PATH_INFO: `/` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:47:04.395838 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 521/1035] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 1 14:47:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 14:47:04.400561 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0eec44-5f8a-472d-b698-5dafe850e105 None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:47:04.400871 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0eec44-5f8a-472d-b698-5dafe850e105 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:47:04.401120 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0eec44-5f8a-472d-b698-5dafe850e105 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:47:04.419696 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-bf0eec44-5f8a-472d-b698-5dafe850e105 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:47:04.420882 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-bf0eec44-5f8a-472d-b698-5dafe850e105 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:47:04.441014 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8bf919-db99-4bc7-ad9a-c85173c2be9d None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:47:04.476795 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495b9aad-08d3-43fb-b764-14720f0c5b15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:47:04.477398 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495b9aad-08d3-43fb-b764-14720f0c5b15 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:47:04.477678 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495b9aad-08d3-43fb-b764-14720f0c5b15 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:47:04.477946 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495b9aad-08d3-43fb-b764-14720f0c5b15 None admin] PATH_INFO: `/v3/users` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:47:04.478694 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495b9aad-08d3-43fb-b764-14720f0c5b15 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81341) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:47:04.478997 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495b9aad-08d3-43fb-b764-14720f0c5b15 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:47:04.480421 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495b9aad-08d3-43fb-b764-14720f0c5b15 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:47:04.489550 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 522/1036] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 1 14:47:04 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 14:47:04.502959 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 515/1037] 10.222.0.33 () {56 vars in 1133 bytes} [Thu Sep 1 14:47:04 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:47:04.504087 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0eec44-5f8a-472d-b698-5dafe850e105 None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:47:04.553197 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:47:04.603701 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0eec44-5f8a-472d-b698-5dafe850e105 None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:47:04.636566 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799db051-6a3c-4df3-95bb-58f5b41a5ea9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:47:04.637158 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799db051-6a3c-4df3-95bb-58f5b41a5ea9 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:47:04.637433 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799db051-6a3c-4df3-95bb-58f5b41a5ea9 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:47:04.637698 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799db051-6a3c-4df3-95bb-58f5b41a5ea9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:47:04.714621 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799db051-6a3c-4df3-95bb-58f5b41a5ea9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:47:04.716060 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799db051-6a3c-4df3-95bb-58f5b41a5ea9 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:47:04.761842 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495b9aad-08d3-43fb-b764-14720f0c5b15 None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:47:04.764344 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:47:04.791408 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:47:04.826945 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 523/1038] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Sep 1 14:47:04 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 14:47:04.828057 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799db051-6a3c-4df3-95bb-58f5b41a5ea9 None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:47:04.875442 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495b9aad-08d3-43fb-b764-14720f0c5b15 None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:47:04.911784 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cc6a68-9dca-4ca3-80d0-040c1a85db0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:47:04.912409 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cc6a68-9dca-4ca3-80d0-040c1a85db0b None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:47:04.912686 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cc6a68-9dca-4ca3-80d0-040c1a85db0b None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:47:04.912952 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cc6a68-9dca-4ca3-80d0-040c1a85db0b None admin] PATH_INFO: `/v3/domains/default` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:47:04.914324 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cc6a68-9dca-4ca3-80d0-040c1a85db0b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:47:04.915505 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cc6a68-9dca-4ca3-80d0-040c1a85db0b None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:47:04.917051 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 516/1039] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 1 14:47:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:47:04.919726 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7d0206-0662-468e-8a0d-d7b59d2ba12d None None] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:47:04.920509 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7d0206-0662-468e-8a0d-d7b59d2ba12d None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:47:04.921248 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7d0206-0662-468e-8a0d-d7b59d2ba12d None None] PATH_INFO: `/` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:47:04.922679 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 517/1040] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 1 14:47:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:47:04.923545 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7d0206-0662-468e-8a0d-d7b59d2ba12d None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:47:04.926075 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799db051-6a3c-4df3-95bb-58f5b41a5ea9 None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:47:04.961168 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b627ebb6-aa67-4769-b6f8-b90de853e3fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:47:04.961784 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b627ebb6-aa67-4769-b6f8-b90de853e3fd None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:47:04.962058 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b627ebb6-aa67-4769-b6f8-b90de853e3fd None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:47:04.962356 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b627ebb6-aa67-4769-b6f8-b90de853e3fd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:47:04.972056 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b627ebb6-aa67-4769-b6f8-b90de853e3fd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:47:04.973645 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b627ebb6-aa67-4769-b6f8-b90de853e3fd None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:47:04.987735 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application [None req-b627ebb6-aa67-4769-b6f8-b90de853e3fd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 14:47:04.987735 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:47:04.987735 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:47:04.987735 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:47:04.987735 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:47:04.987735 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:47:04.987735 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:47:04.987735 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:47:04.987735 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:47:04.987735 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:47:04.987735 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:47:04.987735 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:47:04.987735 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 14:47:04.987735 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 14:47:04.987735 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 14:47:04.987735 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 14:47:04.989002 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:47:04.989002 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:47:04.989002 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:47:04.989002 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:47:04.989002 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:47:04.989002 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:47:04.989002 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:47:04.989002 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application with Lock( Sep 01 14:47:04.989002 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:47:04.989002 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:47:04.989002 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:47:04.989002 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:47:04.989002 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:47:04.989002 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:47:04.989002 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:47:04.989002 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:47:04.989002 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 14:47:04.990008 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 14:47:04.990008 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 14:47:04.990008 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Sep 01 14:47:04.990008 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 524/1041] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 1 14:47:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:47:04.990008 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9271b45c-1d1d-4a08-b49f-eca020b8252d None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:47:04.990357 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9271b45c-1d1d-4a08-b49f-eca020b8252d None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:47:04.990431 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9271b45c-1d1d-4a08-b49f-eca020b8252d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:47:04.996218 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:47:05.008841 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-9271b45c-1d1d-4a08-b49f-eca020b8252d None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:47:05.010008 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-9271b45c-1d1d-4a08-b49f-eca020b8252d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:47:05.019593 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:47:05.028048 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7d0206-0662-468e-8a0d-d7b59d2ba12d None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:47:05.061033 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ed723b-ee66-4642-9fde-80e0e95255d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:47:05.061627 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ed723b-ee66-4642-9fde-80e0e95255d9 None admin] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:47:05.061906 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ed723b-ee66-4642-9fde-80e0e95255d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:47:05.062175 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ed723b-ee66-4642-9fde-80e0e95255d9 None admin] PATH_INFO: `/v3/users` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:47:05.062839 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ed723b-ee66-4642-9fde-80e0e95255d9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:47:05.064270 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ed723b-ee66-4642-9fde-80e0e95255d9 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:47:05.077517 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 525/1042] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 1 14:47:04 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 14:47:05.078478 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9271b45c-1d1d-4a08-b49f-eca020b8252d None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:47:05.093389 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 518/1043] 10.222.0.33 () {60 vars in 1132 bytes} [Thu Sep 1 14:47:04 2022] POST /identity/v3/users => generated 285 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:47:05.186722 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9271b45c-1d1d-4a08-b49f-eca020b8252d None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:47:05.219065 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:47:05.226696 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f4c3bb-20a1-4560-92cb-71af5e72915b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:47:05.227316 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f4c3bb-20a1-4560-92cb-71af5e72915b None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:47:05.227594 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f4c3bb-20a1-4560-92cb-71af5e72915b None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:47:05.228531 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f4c3bb-20a1-4560-92cb-71af5e72915b None admin] PATH_INFO: `/v3/domains` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:47:05.228531 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f4c3bb-20a1-4560-92cb-71af5e72915b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:47:05.228797 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f4c3bb-20a1-4560-92cb-71af5e72915b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:47:05.230193 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f4c3bb-20a1-4560-92cb-71af5e72915b None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:47:05.243444 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 526/1044] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Sep 1 14:47:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:47:05.249208 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ed723b-ee66-4642-9fde-80e0e95255d9 None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:47:05.349662 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f4c3bb-20a1-4560-92cb-71af5e72915b None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:47:05.353056 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ed723b-ee66-4642-9fde-80e0e95255d9 None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:47:05.400820 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec34944-daa4-4100-a988-fb03c6b43b6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:47:05.401758 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec34944-daa4-4100-a988-fb03c6b43b6e None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:47:05.402041 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec34944-daa4-4100-a988-fb03c6b43b6e None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:47:05.402340 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec34944-daa4-4100-a988-fb03c6b43b6e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:47:05.409105 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec34944-daa4-4100-a988-fb03c6b43b6e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:47:05.410621 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec34944-daa4-4100-a988-fb03c6b43b6e None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:47:05.428820 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application [None req-1ec34944-daa4-4100-a988-fb03c6b43b6e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 14:47:05.428820 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:47:05.428820 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:47:05.428820 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:47:05.428820 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:47:05.428820 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:47:05.428820 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:47:05.428820 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:47:05.428820 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:47:05.428820 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:47:05.428820 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:47:05.428820 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:47:05.428820 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 14:47:05.428820 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 14:47:05.428820 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 14:47:05.429824 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 14:47:05.429824 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:47:05.429824 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:47:05.429824 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:47:05.429824 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:47:05.429824 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:47:05.429824 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:47:05.429824 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:47:05.429824 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application with Lock( Sep 01 14:47:05.429824 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:47:05.429824 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:47:05.429824 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:47:05.429824 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:47:05.429824 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:47:05.429824 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:47:05.429824 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:47:05.429824 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:47:05.430824 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 14:47:05.430824 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 14:47:05.430824 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 14:47:05.430824 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 14:47:05.430824 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 14:47:05.430824 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 14:47:05.430824 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 14:47:05.430824 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Sep 01 14:47:05.434913 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 519/1045] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Sep 1 14:47:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:47:05.435790 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec34944-daa4-4100-a988-fb03c6b43b6e None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:47:05.449468 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:47:05.461124 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f4c3bb-20a1-4560-92cb-71af5e72915b None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:47:05.498047 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cc8453-94d4-43c7-b272-fc3f3031d97e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:47:05.498709 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cc8453-94d4-43c7-b272-fc3f3031d97e None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:47:05.498989 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cc8453-94d4-43c7-b272-fc3f3031d97e None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:47:05.499261 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cc8453-94d4-43c7-b272-fc3f3031d97e None admin] PATH_INFO: `/v3/services/image` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:47:05.499840 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cc8453-94d4-43c7-b272-fc3f3031d97e None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:47:05.501195 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cc8453-94d4-43c7-b272-fc3f3031d97e None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:47:05.510500 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application [None req-96cc8453-94d4-43c7-b272-fc3f3031d97e None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 01 14:47:05.510500 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:47:05.510500 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 14:47:05.510500 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 14:47:05.510500 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 14:47:05.510500 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 14:47:05.510500 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 14:47:05.510500 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 14:47:05.510500 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 01 14:47:05.510500 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Sep 01 14:47:05.510500 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 14:47:05.510500 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Sep 01 14:47:05.510500 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:47:05.510500 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:47:05.510500 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:47:05.510500 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:47:05.510500 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:47:05.511525 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:47:05.511525 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:47:05.511525 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:47:05.511525 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:47:05.511525 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:47:05.511525 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:47:05.511525 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 14:47:05.511525 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 14:47:05.511525 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 14:47:05.511525 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 14:47:05.511525 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:47:05.511525 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:47:05.511525 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:47:05.511525 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:47:05.511525 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:47:05.511525 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:47:05.512414 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:47:05.512414 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application with Lock( Sep 01 14:47:05.512414 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:47:05.512414 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:47:05.512414 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:47:05.512414 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:47:05.512414 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:47:05.512414 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:47:05.512414 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:47:05.512414 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:47:05.512414 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 14:47:05.512414 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 14:47:05.512414 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 01 14:47:05.512414 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Sep 01 14:47:05.512414 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 527/1046] 10.222.0.33 () {56 vars in 1105 bytes} [Thu Sep 1 14:47:05 2022] GET /identity/v3/services/image => generated 86 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 14:47:05.519467 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cc8453-94d4-43c7-b272-fc3f3031d97e None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:47:05.546696 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec34944-daa4-4100-a988-fb03c6b43b6e None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:47:05.577217 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:47:05.583744 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ace584-2631-4e90-a35c-0331c00a2b46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:47:05.584353 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ace584-2631-4e90-a35c-0331c00a2b46 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:47:05.584625 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ace584-2631-4e90-a35c-0331c00a2b46 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:47:05.584894 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ace584-2631-4e90-a35c-0331c00a2b46 None admin] PATH_INFO: `/v3/projects` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:47:05.585583 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ace584-2631-4e90-a35c-0331c00a2b46 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81341) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:47:05.585880 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ace584-2631-4e90-a35c-0331c00a2b46 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:47:05.587348 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ace584-2631-4e90-a35c-0331c00a2b46 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:47:05.600869 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 520/1047] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Sep 1 14:47:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:47:05.607271 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ace584-2631-4e90-a35c-0331c00a2b46 None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:47:05.626522 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cc8453-94d4-43c7-b272-fc3f3031d97e None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:47:05.671113 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f2caa8-fa7c-4ca5-8581-eb329c88a655 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:47:05.671903 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f2caa8-fa7c-4ca5-8581-eb329c88a655 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:47:05.672406 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f2caa8-fa7c-4ca5-8581-eb329c88a655 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:47:05.672834 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f2caa8-fa7c-4ca5-8581-eb329c88a655 None admin] PATH_INFO: `/v3/services` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:47:05.673513 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f2caa8-fa7c-4ca5-8581-eb329c88a655 None admin] RBAC: Adding query filter params (name=image) {{(pid=81342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:47:05.673817 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f2caa8-fa7c-4ca5-8581-eb329c88a655 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:47:05.675777 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f2caa8-fa7c-4ca5-8581-eb329c88a655 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:47:05.680654 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:47:05.681077 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 528/1048] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 1 14:47:05 2022] GET /identity/v3/services?name=image => generated 122 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 14:47:05.686696 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f2caa8-fa7c-4ca5-8581-eb329c88a655 None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:47:05.714495 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ace584-2631-4e90-a35c-0331c00a2b46 None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:47:05.750036 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fa533d-b84e-4226-b5a6-9eaaeade0a7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:47:05.750698 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fa533d-b84e-4226-b5a6-9eaaeade0a7e None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:47:05.750951 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fa533d-b84e-4226-b5a6-9eaaeade0a7e None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:47:05.751221 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fa533d-b84e-4226-b5a6-9eaaeade0a7e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:47:05.751939 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44fa533d-b84e-4226-b5a6-9eaaeade0a7e None admin] RBAC: Adding query filter params (role.id=050916e5d27c476c9eed7850b136dd98, scope.project.id=e8c85b4bd17e4e89b56bf0f831b474d5, user.id=4858bb8621854ec8aba836e7d9050498) {{(pid=81341) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:47:05.752275 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44fa533d-b84e-4226-b5a6-9eaaeade0a7e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:47:05.753751 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44fa533d-b84e-4226-b5a6-9eaaeade0a7e None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:47:05.766672 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 521/1049] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Sep 1 14:47:05 2022] GET /identity/v3/role_assignments?user.id=4858bb8621854ec8aba836e7d9050498&scope.project.id=e8c85b4bd17e4e89b56bf0f831b474d5&role.id=050916e5d27c476c9eed7850b136dd98 => generated 259 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:47:05.795916 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f2caa8-fa7c-4ca5-8581-eb329c88a655 None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:47:05.819038 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:47:05.835450 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faea0719-db8b-4474-a907-3d7eea9478c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:47:05.836076 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faea0719-db8b-4474-a907-3d7eea9478c7 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:47:05.836366 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faea0719-db8b-4474-a907-3d7eea9478c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:47:05.836618 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faea0719-db8b-4474-a907-3d7eea9478c7 None admin] PATH_INFO: `/v3/services` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:47:05.837264 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faea0719-db8b-4474-a907-3d7eea9478c7 None admin] RBAC: Adding query filter params (type=image) {{(pid=81342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:47:05.837563 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faea0719-db8b-4474-a907-3d7eea9478c7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:47:05.838998 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faea0719-db8b-4474-a907-3d7eea9478c7 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:47:05.849842 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 529/1050] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 1 14:47:05 2022] GET /identity/v3/services?type=image => generated 350 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 14:47:05.851215 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fa533d-b84e-4226-b5a6-9eaaeade0a7e None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:47:05.955835 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fa533d-b84e-4226-b5a6-9eaaeade0a7e None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:47:05.990528 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ec54dc-7e44-4242-9107-cd16805f1bef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:47:05.991051 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ec54dc-7e44-4242-9107-cd16805f1bef None admin] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:47:05.991324 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ec54dc-7e44-4242-9107-cd16805f1bef None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:47:05.991589 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ec54dc-7e44-4242-9107-cd16805f1bef None admin] PATH_INFO: `/v3/endpoints` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:47:05.992251 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ec54dc-7e44-4242-9107-cd16805f1bef None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:47:05.993615 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ec54dc-7e44-4242-9107-cd16805f1bef None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:47:05.998039 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:47:06.016422 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 522/1051] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Sep 1 14:47:05 2022] POST /identity/v3/endpoints => generated 327 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:47:07.043609 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b33dca0-d5d4-440a-8b00-86ce7f0b6e32 None None] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:47:07.044098 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b33dca0-d5d4-440a-8b00-86ce7f0b6e32 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:47:07.044098 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b33dca0-d5d4-440a-8b00-86ce7f0b6e32 None None] PATH_INFO: `/` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:47:07.045541 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 530/1052] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 1 14:47:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:47:07.050999 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5aa31c1-32b4-4505-a6c3-6afd6903d2f2 None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:47:07.051378 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5aa31c1-32b4-4505-a6c3-6afd6903d2f2 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:47:07.051538 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5aa31c1-32b4-4505-a6c3-6afd6903d2f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:47:07.070334 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-d5aa31c1-32b4-4505-a6c3-6afd6903d2f2 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:47:07.071529 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-d5aa31c1-32b4-4505-a6c3-6afd6903d2f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:47:07.144765 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 523/1053] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 1 14:47:07 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 14:47:07.410026 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b33dca0-d5d4-440a-8b00-86ce7f0b6e32 None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:47:07.515004 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b33dca0-d5d4-440a-8b00-86ce7f0b6e32 None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:47:07.549948 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf086ef-df70-4a98-bcc2-0a90f712821f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:47:07.551011 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf086ef-df70-4a98-bcc2-0a90f712821f None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:47:07.551509 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf086ef-df70-4a98-bcc2-0a90f712821f None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:47:07.551978 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf086ef-df70-4a98-bcc2-0a90f712821f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:47:07.562324 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf086ef-df70-4a98-bcc2-0a90f712821f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 14:47:07.567409 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application [None req-ebf086ef-df70-4a98-bcc2-0a90f712821f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 14:47:07.567409 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:47:07.567409 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:47:07.567409 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:47:07.567409 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:47:07.567409 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:47:07.567409 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:47:07.567409 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:47:07.567409 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:47:07.567409 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:47:07.567409 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:47:07.567409 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:47:07.567409 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 14:47:07.567409 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 14:47:07.567409 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 14:47:07.567409 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 14:47:07.568358 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 14:47:07.568358 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 14:47:07.568358 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 14:47:07.568358 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 14:47:07.568358 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:47:07.568358 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:47:07.568358 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:47:07.568358 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:47:07.568358 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:47:07.568358 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:47:07.568358 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:47:07.568358 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application with Lock( Sep 01 14:47:07.568358 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:47:07.568358 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:47:07.568358 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:47:07.568358 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:47:07.569244 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:47:07.569244 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:47:07.569244 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:47:07.569244 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:47:07.569244 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 14:47:07.569244 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 14:47:07.569244 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 14:47:07.569244 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 14:47:07.569244 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 14:47:07.569244 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 14:47:07.569244 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 14:47:07.569244 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Sep 01 14:47:07.571433 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 531/1054] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Sep 1 14:47:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:47:07.577227 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5aa31c1-32b4-4505-a6c3-6afd6903d2f2 None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:47:07.653564 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec87445-de89-4bc5-94ad-3812a3565c72 None None] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:47:07.653755 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec87445-de89-4bc5-94ad-3812a3565c72 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:47:07.654008 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec87445-de89-4bc5-94ad-3812a3565c72 None None] PATH_INFO: `/` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:47:07.655890 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 532/1055] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 1 14:47:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:47:07.661792 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af793059-490c-430e-aa3f-d622cc8ce6e9 None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:47:07.661792 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af793059-490c-430e-aa3f-d622cc8ce6e9 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:47:07.661984 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af793059-490c-430e-aa3f-d622cc8ce6e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:47:07.681855 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-af793059-490c-430e-aa3f-d622cc8ce6e9 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:47:07.683089 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-af793059-490c-430e-aa3f-d622cc8ce6e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:47:07.693495 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5aa31c1-32b4-4505-a6c3-6afd6903d2f2 None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:47:07.713178 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:47:07.726815 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:47:07.736680 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfeaad6-5fb1-4ad2-a7dd-f03849f0a902 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:47:07.737596 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfeaad6-5fb1-4ad2-a7dd-f03849f0a902 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:47:07.737872 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfeaad6-5fb1-4ad2-a7dd-f03849f0a902 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:47:07.738139 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfeaad6-5fb1-4ad2-a7dd-f03849f0a902 None admin] PATH_INFO: `/v3/roles` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:47:07.738826 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfeaad6-5fb1-4ad2-a7dd-f03849f0a902 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81341) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:47:07.739123 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfeaad6-5fb1-4ad2-a7dd-f03849f0a902 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:47:07.741157 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfeaad6-5fb1-4ad2-a7dd-f03849f0a902 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:47:07.749881 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 524/1056] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Sep 1 14:47:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:47:07.755613 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfeaad6-5fb1-4ad2-a7dd-f03849f0a902 None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:47:07.758783 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 533/1057] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 1 14:47:07 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 14:47:07.860862 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfeaad6-5fb1-4ad2-a7dd-f03849f0a902 None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:47:07.895552 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b1e90b-8257-4cb7-9c0b-11ca53b689d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:47:07.896420 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b1e90b-8257-4cb7-9c0b-11ca53b689d6 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:47:07.896953 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b1e90b-8257-4cb7-9c0b-11ca53b689d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:47:07.897371 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b1e90b-8257-4cb7-9c0b-11ca53b689d6 None admin] PATH_INFO: `/v3/users/e7203c765cb446eabeb1d04cfc2e9f89` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:47:07.912173 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b1e90b-8257-4cb7-9c0b-11ca53b689d6 None admin] RBAC: Authorizing `identity:get_user(user_id=e7203c765cb446eabeb1d04cfc2e9f89)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:47:07.913878 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b1e90b-8257-4cb7-9c0b-11ca53b689d6 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:47:07.920275 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 525/1058] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Sep 1 14:47:07 2022] GET /identity/v3/users/e7203c765cb446eabeb1d04cfc2e9f89 => generated 285 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 14:47:07.921940 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af793059-490c-430e-aa3f-d622cc8ce6e9 None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:47:07.934852 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2436ac3-e7fa-4c4a-a7e4-cddff4443cad None None] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:47:07.935140 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2436ac3-e7fa-4c4a-a7e4-cddff4443cad None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:47:07.935538 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2436ac3-e7fa-4c4a-a7e4-cddff4443cad None None] PATH_INFO: `/` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:47:07.936738 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 526/1059] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 1 14:47:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 14:47:07.942985 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c81e479-1f03-4b15-aa8f-d3654412427d None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:47:07.943266 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c81e479-1f03-4b15-aa8f-d3654412427d None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:47:07.943535 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c81e479-1f03-4b15-aa8f-d3654412427d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:47:07.967653 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-1c81e479-1f03-4b15-aa8f-d3654412427d None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:47:07.969945 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-1c81e479-1f03-4b15-aa8f-d3654412427d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:47:08.000360 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:47:08.030917 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af793059-490c-430e-aa3f-d622cc8ce6e9 None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:47:08.042267 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 527/1060] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 1 14:47:07 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 14:47:08.043253 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c81e479-1f03-4b15-aa8f-d3654412427d None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:47:08.067647 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d6cc7d-3faa-454b-a0d8-381e085bc902 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:47:08.068249 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d6cc7d-3faa-454b-a0d8-381e085bc902 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:47:08.068533 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d6cc7d-3faa-454b-a0d8-381e085bc902 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:47:08.068805 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d6cc7d-3faa-454b-a0d8-381e085bc902 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:47:08.115813 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:47:08.145831 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d6cc7d-3faa-454b-a0d8-381e085bc902 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:47:08.147499 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d6cc7d-3faa-454b-a0d8-381e085bc902 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:47:08.149109 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c81e479-1f03-4b15-aa8f-d3654412427d None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:47:08.183810 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a44a26f-fcf1-49e9-ae2f-cc2e29e29876 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:47:08.184376 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a44a26f-fcf1-49e9-ae2f-cc2e29e29876 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:47:08.184657 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a44a26f-fcf1-49e9-ae2f-cc2e29e29876 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:47:08.184929 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a44a26f-fcf1-49e9-ae2f-cc2e29e29876 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:47:08.195877 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a44a26f-fcf1-49e9-ae2f-cc2e29e29876 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 14:47:08.200633 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application [None req-9a44a26f-fcf1-49e9-ae2f-cc2e29e29876 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 01 14:47:08.200633 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:47:08.200633 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:47:08.200633 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:47:08.200633 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:47:08.200633 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:47:08.200633 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:47:08.200633 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:47:08.200633 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:47:08.200633 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:47:08.200633 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:47:08.200633 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:47:08.200633 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 14:47:08.200633 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 14:47:08.200633 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 14:47:08.200633 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 14:47:08.201596 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 14:47:08.201596 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 14:47:08.201596 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 14:47:08.201596 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 14:47:08.201596 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:47:08.201596 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:47:08.201596 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:47:08.201596 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:47:08.201596 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:47:08.201596 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:47:08.201596 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:47:08.201596 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application with Lock( Sep 01 14:47:08.201596 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:47:08.201596 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:47:08.201596 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:47:08.201596 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:47:08.202500 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:47:08.202500 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:47:08.202500 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:47:08.202500 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:47:08.202500 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 14:47:08.202500 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 14:47:08.202500 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 14:47:08.202500 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 14:47:08.202500 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 14:47:08.202500 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 14:47:08.202500 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 01 14:47:08.202500 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Sep 01 14:47:08.202500 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 528/1061] 10.222.0.33 () {56 vars in 1120 bytes} [Thu Sep 1 14:47:08 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:47:08.207989 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a44a26f-fcf1-49e9-ae2f-cc2e29e29876 None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:47:08.235130 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:47:08.260417 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 534/1062] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Sep 1 14:47:07 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 14:47:08.266075 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d6cc7d-3faa-454b-a0d8-381e085bc902 None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:47:08.323924 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a44a26f-fcf1-49e9-ae2f-cc2e29e29876 None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:47:08.342624 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:47:08.357891 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845be581-fe37-4942-a3ca-efb41a433384 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:47:08.358524 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845be581-fe37-4942-a3ca-efb41a433384 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:47:08.358806 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845be581-fe37-4942-a3ca-efb41a433384 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:47:08.359080 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845be581-fe37-4942-a3ca-efb41a433384 None admin] PATH_INFO: `/v3/roles` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:47:08.359742 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845be581-fe37-4942-a3ca-efb41a433384 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81341) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:47:08.360042 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845be581-fe37-4942-a3ca-efb41a433384 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:47:08.361428 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845be581-fe37-4942-a3ca-efb41a433384 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:47:08.367568 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d6cc7d-3faa-454b-a0d8-381e085bc902 None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:47:08.369662 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 529/1063] 10.222.0.33 () {56 vars in 1115 bytes} [Thu Sep 1 14:47:08 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 14:47:08.370677 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845be581-fe37-4942-a3ca-efb41a433384 None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:47:08.400758 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07866186-c7b3-40b3-86e2-38f4853efd75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:47:08.401376 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07866186-c7b3-40b3-86e2-38f4853efd75 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:47:08.401649 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07866186-c7b3-40b3-86e2-38f4853efd75 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:47:08.401915 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07866186-c7b3-40b3-86e2-38f4853efd75 None admin] PATH_INFO: `/v3/projects/1f8ecb764c6a437bb0911fe60a148b9c` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:47:08.403042 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07866186-c7b3-40b3-86e2-38f4853efd75 None admin] RBAC: Authorizing `identity:get_project(project_id=1f8ecb764c6a437bb0911fe60a148b9c)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:47:08.404449 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07866186-c7b3-40b3-86e2-38f4853efd75 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:47:08.406362 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 535/1064] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Sep 1 14:47:08 2022] GET /identity/v3/projects/1f8ecb764c6a437bb0911fe60a148b9c => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 14:47:08.406997 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07866186-c7b3-40b3-86e2-38f4853efd75 None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:47:08.458141 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:47:08.470701 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845be581-fe37-4942-a3ca-efb41a433384 None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:47:08.503882 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b710836-0929-4922-85e9-2c9b677de44b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:47:08.504476 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b710836-0929-4922-85e9-2c9b677de44b None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:47:08.504757 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b710836-0929-4922-85e9-2c9b677de44b None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:47:08.505026 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b710836-0929-4922-85e9-2c9b677de44b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:47:08.507159 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07866186-c7b3-40b3-86e2-38f4853efd75 None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:47:08.543362 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02552b4-4808-447b-af5a-99d47ba59782 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:47:08.543979 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02552b4-4808-447b-af5a-99d47ba59782 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:47:08.544253 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02552b4-4808-447b-af5a-99d47ba59782 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:47:08.544518 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02552b4-4808-447b-af5a-99d47ba59782 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:47:08.555308 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02552b4-4808-447b-af5a-99d47ba59782 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:47:08.556733 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02552b4-4808-447b-af5a-99d47ba59782 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:47:08.558444 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:47:08.571646 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b710836-0929-4922-85e9-2c9b677de44b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:47:08.572470 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application [None req-b02552b4-4808-447b-af5a-99d47ba59782 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 14:47:08.572470 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:47:08.572470 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:47:08.572470 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:47:08.572470 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:47:08.572470 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:47:08.572470 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:47:08.572470 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:47:08.572470 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:47:08.572470 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:47:08.572470 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:47:08.572470 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:47:08.572470 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 14:47:08.572470 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 14:47:08.572470 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 14:47:08.572470 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 14:47:08.573664 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:47:08.573664 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:47:08.573664 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:47:08.573664 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:47:08.573664 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:47:08.573664 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:47:08.573664 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:47:08.573664 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application with Lock( Sep 01 14:47:08.573664 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:47:08.573664 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:47:08.573664 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:47:08.573664 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:47:08.573664 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:47:08.573664 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:47:08.573664 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:47:08.573664 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:47:08.573664 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 14:47:08.574812 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 14:47:08.574812 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 14:47:08.574812 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Sep 01 14:47:08.575001 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b710836-0929-4922-85e9-2c9b677de44b None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:47:08.578401 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 536/1065] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 1 14:47:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:47:08.579263 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02552b4-4808-447b-af5a-99d47ba59782 None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:47:08.656944 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:47:08.672117 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 530/1066] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Sep 1 14:47:08 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 14:47:08.672941 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b710836-0929-4922-85e9-2c9b677de44b None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:47:08.678732 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02552b4-4808-447b-af5a-99d47ba59782 None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:47:08.711271 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757336b2-9469-4dc6-8f2a-dfd1378fe14f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:47:08.711776 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757336b2-9469-4dc6-8f2a-dfd1378fe14f None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:47:08.712047 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757336b2-9469-4dc6-8f2a-dfd1378fe14f None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:47:08.712310 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757336b2-9469-4dc6-8f2a-dfd1378fe14f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:47:08.713026 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757336b2-9469-4dc6-8f2a-dfd1378fe14f None admin] RBAC: Adding query filter params (role.id=2a2adb769e55476b82e5014897e0a9c0, scope.project.id=1f8ecb764c6a437bb0911fe60a148b9c, user.id=e7203c765cb446eabeb1d04cfc2e9f89) {{(pid=81342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:47:08.713327 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757336b2-9469-4dc6-8f2a-dfd1378fe14f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:47:08.714783 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757336b2-9469-4dc6-8f2a-dfd1378fe14f None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:47:08.719922 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 537/1067] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Sep 1 14:47:08 2022] GET /identity/v3/role_assignments?user.id=e7203c765cb446eabeb1d04cfc2e9f89&scope.project.id=1f8ecb764c6a437bb0911fe60a148b9c&role.id=2a2adb769e55476b82e5014897e0a9c0 => generated 259 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:47:08.720677 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757336b2-9469-4dc6-8f2a-dfd1378fe14f None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:47:08.785672 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b710836-0929-4922-85e9-2c9b677de44b None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:47:08.786754 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:47:08.823633 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed84145-a0d3-4677-b8d5-98ca38e9cc3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:47:08.824316 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed84145-a0d3-4677-b8d5-98ca38e9cc3c None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:47:08.824547 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed84145-a0d3-4677-b8d5-98ca38e9cc3c None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:47:08.824812 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed84145-a0d3-4677-b8d5-98ca38e9cc3c None admin] PATH_INFO: `/v3/domains` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:47:08.825740 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed84145-a0d3-4677-b8d5-98ca38e9cc3c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81341) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:47:08.825740 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed84145-a0d3-4677-b8d5-98ca38e9cc3c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:47:08.827212 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed84145-a0d3-4677-b8d5-98ca38e9cc3c None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:47:08.834914 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757336b2-9469-4dc6-8f2a-dfd1378fe14f None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:47:08.845338 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 531/1068] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Sep 1 14:47:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:47:08.846525 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed84145-a0d3-4677-b8d5-98ca38e9cc3c None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:47:08.887958 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8397d4da-07ee-42b9-b371-c6d8a860b648 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:47:08.888595 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8397d4da-07ee-42b9-b371-c6d8a860b648 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:47:08.888873 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8397d4da-07ee-42b9-b371-c6d8a860b648 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:47:08.889143 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8397d4da-07ee-42b9-b371-c6d8a860b648 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:47:08.899909 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:47:08.900969 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8397d4da-07ee-42b9-b371-c6d8a860b648 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:47:08.902469 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8397d4da-07ee-42b9-b371-c6d8a860b648 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:47:08.917591 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application [None req-8397d4da-07ee-42b9-b371-c6d8a860b648 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 14:47:08.917591 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:47:08.917591 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:47:08.917591 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:47:08.917591 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:47:08.917591 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:47:08.917591 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:47:08.917591 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:47:08.917591 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:47:08.917591 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:47:08.917591 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:47:08.917591 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:47:08.917591 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 14:47:08.917591 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 14:47:08.917591 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 14:47:08.917591 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 14:47:08.918667 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:47:08.918667 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:47:08.918667 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:47:08.918667 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:47:08.918667 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:47:08.918667 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:47:08.918667 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:47:08.918667 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application with Lock( Sep 01 14:47:08.918667 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:47:08.918667 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:47:08.918667 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:47:08.918667 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:47:08.918667 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:47:08.918667 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:47:08.918667 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:47:08.918667 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:47:08.918667 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 14:47:08.919628 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 14:47:08.919628 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 14:47:08.919628 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Sep 01 14:47:08.919628 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 538/1069] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 1 14:47:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:47:08.924682 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8397d4da-07ee-42b9-b371-c6d8a860b648 None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:47:08.960634 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed84145-a0d3-4677-b8d5-98ca38e9cc3c None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:47:08.999527 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2bb8cd-936d-4d34-b5b4-c97def25dda5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:47:09.000078 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2bb8cd-936d-4d34-b5b4-c97def25dda5 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:47:09.000354 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2bb8cd-936d-4d34-b5b4-c97def25dda5 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:47:09.000622 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2bb8cd-936d-4d34-b5b4-c97def25dda5 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:47:09.009526 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2bb8cd-936d-4d34-b5b4-c97def25dda5 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:47:09.011018 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2bb8cd-936d-4d34-b5b4-c97def25dda5 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:47:09.022964 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application [None req-2a2bb8cd-936d-4d34-b5b4-c97def25dda5 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 01 14:47:09.022964 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:47:09.022964 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:47:09.022964 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:47:09.022964 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:47:09.022964 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:47:09.022964 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:47:09.022964 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:47:09.022964 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:47:09.022964 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:47:09.022964 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:47:09.022964 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:47:09.022964 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 14:47:09.022964 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 14:47:09.022964 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 14:47:09.022964 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 14:47:09.024115 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:47:09.024115 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:47:09.024115 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 14:47:09.024115 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 14:47:09.024115 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 14:47:09.024115 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 14:47:09.024115 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:47:09.024115 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:47:09.024115 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:47:09.024115 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:47:09.024115 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:47:09.024115 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application with Lock( Sep 01 14:47:09.024115 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:47:09.024115 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:47:09.024115 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:47:09.024115 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:47:09.024115 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:47:09.024115 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:47:09.025481 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:47:09.025481 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:47:09.025481 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 14:47:09.025481 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 14:47:09.025481 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 14:47:09.025481 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 14:47:09.025481 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 14:47:09.025481 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 14:47:09.025481 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 01 14:47:09.025481 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Sep 01 14:47:09.028908 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 532/1070] 10.222.0.33 () {56 vars in 1093 bytes} [Thu Sep 1 14:47:08 2022] GET /identity/v3/users/nova => generated 82 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:47:09.029734 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2bb8cd-936d-4d34-b5b4-c97def25dda5 None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:47:09.034984 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8397d4da-07ee-42b9-b371-c6d8a860b648 None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:47:09.047363 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:47:09.073347 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c32b742-6025-4403-95d8-e6d6e596abfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:47:09.073988 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c32b742-6025-4403-95d8-e6d6e596abfa None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:47:09.074220 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c32b742-6025-4403-95d8-e6d6e596abfa None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:47:09.074523 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c32b742-6025-4403-95d8-e6d6e596abfa None admin] PATH_INFO: `/v3/domains` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:47:09.075332 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c32b742-6025-4403-95d8-e6d6e596abfa None admin] RBAC: Adding query filter params (name=Default) {{(pid=81342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:47:09.075636 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c32b742-6025-4403-95d8-e6d6e596abfa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:47:09.077046 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c32b742-6025-4403-95d8-e6d6e596abfa None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:47:09.090605 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 539/1071] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Sep 1 14:47:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 14:47:09.153226 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2bb8cd-936d-4d34-b5b4-c97def25dda5 None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:47:09.165000 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:47:09.192293 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fa7701-5202-4d45-aef4-449f40e5730b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:47:09.193326 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fa7701-5202-4d45-aef4-449f40e5730b None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:47:09.193813 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fa7701-5202-4d45-aef4-449f40e5730b None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:47:09.194320 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fa7701-5202-4d45-aef4-449f40e5730b None admin] PATH_INFO: `/v3/users` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:47:09.195529 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fa7701-5202-4d45-aef4-449f40e5730b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81341) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:47:09.196059 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fa7701-5202-4d45-aef4-449f40e5730b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:47:09.197815 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fa7701-5202-4d45-aef4-449f40e5730b None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:47:09.218231 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 533/1072] 10.222.0.33 () {56 vars in 1133 bytes} [Thu Sep 1 14:47:09 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:47:09.224817 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c32b742-6025-4403-95d8-e6d6e596abfa None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:47:09.335172 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c32b742-6025-4403-95d8-e6d6e596abfa None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:47:09.369620 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555f5f18-d60e-4bd5-b524-bef383737079 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:47:09.370275 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555f5f18-d60e-4bd5-b524-bef383737079 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:47:09.370545 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555f5f18-d60e-4bd5-b524-bef383737079 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:47:09.370809 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555f5f18-d60e-4bd5-b524-bef383737079 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:47:09.381812 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555f5f18-d60e-4bd5-b524-bef383737079 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:47:09.383275 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555f5f18-d60e-4bd5-b524-bef383737079 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:47:09.404864 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application [None req-555f5f18-d60e-4bd5-b524-bef383737079 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 14:47:09.404864 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:47:09.404864 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:47:09.404864 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:47:09.404864 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:47:09.404864 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:47:09.404864 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:47:09.404864 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:47:09.404864 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:47:09.404864 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:47:09.404864 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:47:09.404864 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:47:09.404864 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 14:47:09.404864 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 14:47:09.404864 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 14:47:09.404864 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 14:47:09.406082 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:47:09.406082 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:47:09.406082 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:47:09.406082 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:47:09.406082 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:47:09.406082 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:47:09.406082 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:47:09.406082 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application with Lock( Sep 01 14:47:09.406082 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:47:09.406082 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:47:09.406082 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:47:09.406082 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:47:09.406082 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:47:09.406082 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:47:09.406082 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:47:09.406082 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:47:09.406082 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 14:47:09.407057 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 14:47:09.407057 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 14:47:09.407057 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Sep 01 14:47:09.407057 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 540/1073] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 1 14:47:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:47:09.412074 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fa7701-5202-4d45-aef4-449f40e5730b None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:47:09.508680 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fa7701-5202-4d45-aef4-449f40e5730b None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:47:09.541190 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f2abed-1b5b-4fc7-b984-8c335ca75034 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:47:09.541785 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f2abed-1b5b-4fc7-b984-8c335ca75034 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:47:09.542134 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f2abed-1b5b-4fc7-b984-8c335ca75034 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:47:09.542363 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f2abed-1b5b-4fc7-b984-8c335ca75034 None admin] PATH_INFO: `/v3/domains` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:47:09.542997 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f2abed-1b5b-4fc7-b984-8c335ca75034 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81341) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:47:09.543294 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f2abed-1b5b-4fc7-b984-8c335ca75034 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:47:09.544670 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f2abed-1b5b-4fc7-b984-8c335ca75034 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:47:09.555702 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 534/1074] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Sep 1 14:47:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:47:09.561374 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555f5f18-d60e-4bd5-b524-bef383737079 None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:47:09.586972 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:47:09.663510 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555f5f18-d60e-4bd5-b524-bef383737079 None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:47:09.696767 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f593fe-35bb-45ab-8e43-70f37237df0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:47:09.697413 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f593fe-35bb-45ab-8e43-70f37237df0d None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:47:09.697688 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f593fe-35bb-45ab-8e43-70f37237df0d None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:47:09.697957 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f593fe-35bb-45ab-8e43-70f37237df0d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:47:09.704475 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f593fe-35bb-45ab-8e43-70f37237df0d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:47:09.705917 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f593fe-35bb-45ab-8e43-70f37237df0d None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:47:09.717405 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application [None req-08f593fe-35bb-45ab-8e43-70f37237df0d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 14:47:09.717405 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:47:09.717405 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:47:09.717405 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:47:09.717405 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:47:09.717405 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:47:09.717405 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:47:09.717405 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:47:09.717405 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:47:09.717405 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:47:09.717405 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:47:09.717405 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:47:09.717405 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 14:47:09.717405 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 14:47:09.717405 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 14:47:09.718420 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 14:47:09.718420 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:47:09.718420 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:47:09.718420 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:47:09.718420 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:47:09.718420 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:47:09.718420 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:47:09.718420 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:47:09.718420 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application with Lock( Sep 01 14:47:09.718420 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:47:09.718420 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:47:09.718420 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:47:09.718420 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:47:09.718420 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:47:09.718420 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:47:09.718420 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:47:09.718420 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:47:09.719336 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 14:47:09.719336 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 14:47:09.719336 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 14:47:09.719336 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 14:47:09.719336 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 14:47:09.719336 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 14:47:09.719336 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 14:47:09.719336 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Sep 01 14:47:09.719336 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 541/1075] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Sep 1 14:47:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 14:47:09.724468 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f2abed-1b5b-4fc7-b984-8c335ca75034 None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:47:09.752492 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:47:09.826631 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f2abed-1b5b-4fc7-b984-8c335ca75034 None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:47:09.858433 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced1b682-0d79-4f1a-b590-7133373cc988 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:47:09.858967 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced1b682-0d79-4f1a-b590-7133373cc988 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:47:09.859246 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced1b682-0d79-4f1a-b590-7133373cc988 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:47:09.859520 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced1b682-0d79-4f1a-b590-7133373cc988 None admin] PATH_INFO: `/v3/projects` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:47:09.860205 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced1b682-0d79-4f1a-b590-7133373cc988 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81341) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:47:09.860511 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced1b682-0d79-4f1a-b590-7133373cc988 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:47:09.861941 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced1b682-0d79-4f1a-b590-7133373cc988 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:47:09.873977 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 535/1076] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Sep 1 14:47:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:47:09.880565 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f593fe-35bb-45ab-8e43-70f37237df0d None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:47:09.960093 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:47:09.978220 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f593fe-35bb-45ab-8e43-70f37237df0d None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:47:10.011401 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf623ac-cfb8-46f3-b7d4-057ba8f22d55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:47:10.012046 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf623ac-cfb8-46f3-b7d4-057ba8f22d55 None admin] REQUEST_METHOD: `PUT` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:47:10.012323 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf623ac-cfb8-46f3-b7d4-057ba8f22d55 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:47:10.012592 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf623ac-cfb8-46f3-b7d4-057ba8f22d55 None admin] PATH_INFO: `/v3/projects/e8c85b4bd17e4e89b56bf0f831b474d5/users/4858bb8621854ec8aba836e7d9050498/roles/050916e5d27c476c9eed7850b136dd98` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:47:10.014729 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf623ac-cfb8-46f3-b7d4-057ba8f22d55 None admin] RBAC: Authorizing `identity:create_grant(project_id=e8c85b4bd17e4e89b56bf0f831b474d5, user_id=4858bb8621854ec8aba836e7d9050498, role_id=050916e5d27c476c9eed7850b136dd98)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:47:10.016140 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf623ac-cfb8-46f3-b7d4-057ba8f22d55 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:47:10.030417 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 542/1077] 10.222.0.33 () {58 vars in 1439 bytes} [Thu Sep 1 14:47:09 2022] PUT /identity/v3/projects/e8c85b4bd17e4e89b56bf0f831b474d5/users/4858bb8621854ec8aba836e7d9050498/roles/050916e5d27c476c9eed7850b136dd98 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:47:10.593465 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6ace9e-bbb7-4d98-a064-54de3af5cc73 None None] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:47:10.594026 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6ace9e-bbb7-4d98-a064-54de3af5cc73 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:47:10.594026 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6ace9e-bbb7-4d98-a064-54de3af5cc73 None None] PATH_INFO: `/` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:47:10.595368 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 536/1078] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 1 14:47:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:47:10.601229 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed96f47-197b-415f-aba4-bd98eb932b3b None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:47:10.601669 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed96f47-197b-415f-aba4-bd98eb932b3b None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:47:10.601776 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed96f47-197b-415f-aba4-bd98eb932b3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:47:10.620124 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-0ed96f47-197b-415f-aba4-bd98eb932b3b None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:47:10.621291 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-0ed96f47-197b-415f-aba4-bd98eb932b3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:47:10.687078 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 543/1079] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 1 14:47:10 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 14:47:10.950563 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6ace9e-bbb7-4d98-a064-54de3af5cc73 None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:47:11.034176 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:47:11.049335 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6ace9e-bbb7-4d98-a064-54de3af5cc73 None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:47:11.083908 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32a3337-02c8-4894-9cc2-276b00db0435 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:47:11.084540 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32a3337-02c8-4894-9cc2-276b00db0435 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:47:11.084816 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32a3337-02c8-4894-9cc2-276b00db0435 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:47:11.085084 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32a3337-02c8-4894-9cc2-276b00db0435 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:47:11.095170 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32a3337-02c8-4894-9cc2-276b00db0435 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 14:47:11.099909 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application [None req-a32a3337-02c8-4894-9cc2-276b00db0435 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 14:47:11.099909 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:47:11.099909 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:47:11.099909 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:47:11.099909 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:47:11.099909 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:47:11.099909 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:47:11.099909 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:47:11.099909 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:47:11.099909 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:47:11.099909 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:47:11.099909 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:47:11.099909 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 14:47:11.099909 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 14:47:11.099909 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 14:47:11.099909 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 14:47:11.101004 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 14:47:11.101004 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 14:47:11.101004 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 14:47:11.101004 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 14:47:11.101004 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:47:11.101004 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:47:11.101004 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:47:11.101004 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:47:11.101004 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:47:11.101004 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:47:11.101004 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:47:11.101004 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application with Lock( Sep 01 14:47:11.101004 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:47:11.101004 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:47:11.101004 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:47:11.101004 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:47:11.101900 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:47:11.101900 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:47:11.101900 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:47:11.101900 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:47:11.101900 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 14:47:11.101900 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 14:47:11.101900 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 14:47:11.101900 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 14:47:11.101900 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 14:47:11.101900 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 14:47:11.101900 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 14:47:11.101900 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Sep 01 14:47:11.101900 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 537/1080] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Sep 1 14:47:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:47:11.106972 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed96f47-197b-415f-aba4-bd98eb932b3b None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:47:11.174163 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:47:11.204114 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed96f47-197b-415f-aba4-bd98eb932b3b None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:47:11.236518 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9fcede-caed-4b66-8cc0-e480b62c4386 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:47:11.237107 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9fcede-caed-4b66-8cc0-e480b62c4386 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:47:11.237887 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9fcede-caed-4b66-8cc0-e480b62c4386 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:47:11.238161 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9fcede-caed-4b66-8cc0-e480b62c4386 None admin] PATH_INFO: `/v3/roles` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:47:11.238846 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9fcede-caed-4b66-8cc0-e480b62c4386 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:47:11.239141 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9fcede-caed-4b66-8cc0-e480b62c4386 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:47:11.240489 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9fcede-caed-4b66-8cc0-e480b62c4386 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:47:11.248290 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 544/1081] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Sep 1 14:47:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 14:47:11.254438 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32a3337-02c8-4894-9cc2-276b00db0435 None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:47:11.350213 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32a3337-02c8-4894-9cc2-276b00db0435 None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:47:11.383162 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e63271-af21-46cc-b812-7576260c6c96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:47:11.383778 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e63271-af21-46cc-b812-7576260c6c96 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:47:11.384054 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e63271-af21-46cc-b812-7576260c6c96 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:47:11.384411 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e63271-af21-46cc-b812-7576260c6c96 None admin] PATH_INFO: `/v3/users/e7203c765cb446eabeb1d04cfc2e9f89` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:47:11.386302 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e63271-af21-46cc-b812-7576260c6c96 None admin] RBAC: Authorizing `identity:get_user(user_id=e7203c765cb446eabeb1d04cfc2e9f89)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:47:11.387742 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e63271-af21-46cc-b812-7576260c6c96 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:47:11.389457 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 538/1082] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Sep 1 14:47:11 2022] GET /identity/v3/users/e7203c765cb446eabeb1d04cfc2e9f89 => generated 285 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 14:47:11.395573 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9fcede-caed-4b66-8cc0-e480b62c4386 None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:47:11.493309 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9fcede-caed-4b66-8cc0-e480b62c4386 None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:47:11.525685 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54798fc4-1de0-46c9-bbf8-92699627cdff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:47:11.526360 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54798fc4-1de0-46c9-bbf8-92699627cdff None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:47:11.526637 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54798fc4-1de0-46c9-bbf8-92699627cdff None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:47:11.526903 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54798fc4-1de0-46c9-bbf8-92699627cdff None admin] PATH_INFO: `/v3/projects/1f8ecb764c6a437bb0911fe60a148b9c` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:47:11.528004 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54798fc4-1de0-46c9-bbf8-92699627cdff None admin] RBAC: Authorizing `identity:get_project(project_id=1f8ecb764c6a437bb0911fe60a148b9c)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:47:11.529412 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54798fc4-1de0-46c9-bbf8-92699627cdff None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:47:11.531220 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 545/1083] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Sep 1 14:47:11 2022] GET /identity/v3/projects/1f8ecb764c6a437bb0911fe60a148b9c => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:47:11.536955 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e63271-af21-46cc-b812-7576260c6c96 None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:47:11.552813 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:47:11.634915 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e63271-af21-46cc-b812-7576260c6c96 None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:47:11.668155 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c153089-e857-4f11-a4e9-52c526456d69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:47:11.668824 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c153089-e857-4f11-a4e9-52c526456d69 None admin] REQUEST_METHOD: `PUT` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:47:11.669092 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c153089-e857-4f11-a4e9-52c526456d69 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:47:11.669360 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c153089-e857-4f11-a4e9-52c526456d69 None admin] PATH_INFO: `/v3/projects/1f8ecb764c6a437bb0911fe60a148b9c/users/e7203c765cb446eabeb1d04cfc2e9f89/roles/2a2adb769e55476b82e5014897e0a9c0` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:47:11.671240 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c153089-e857-4f11-a4e9-52c526456d69 None admin] RBAC: Authorizing `identity:create_grant(project_id=1f8ecb764c6a437bb0911fe60a148b9c, user_id=e7203c765cb446eabeb1d04cfc2e9f89, role_id=2a2adb769e55476b82e5014897e0a9c0)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:47:11.672668 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c153089-e857-4f11-a4e9-52c526456d69 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:47:11.686098 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 539/1084] 10.222.0.33 () {58 vars in 1439 bytes} [Thu Sep 1 14:47:11 2022] PUT /identity/v3/projects/1f8ecb764c6a437bb0911fe60a148b9c/users/e7203c765cb446eabeb1d04cfc2e9f89/roles/2a2adb769e55476b82e5014897e0a9c0 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:47:11.917689 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5098b54e-0710-4587-a006-2485aecefdc6 None None] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:47:11.918279 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5098b54e-0710-4587-a006-2485aecefdc6 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:47:11.918279 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5098b54e-0710-4587-a006-2485aecefdc6 None None] PATH_INFO: `/` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:47:11.919717 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 546/1085] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 1 14:47:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:47:11.924740 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48687ccc-9c1f-4c59-aead-caf9ea0826fb None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:47:11.925587 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48687ccc-9c1f-4c59-aead-caf9ea0826fb None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:47:11.925865 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48687ccc-9c1f-4c59-aead-caf9ea0826fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:47:11.943626 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-48687ccc-9c1f-4c59-aead-caf9ea0826fb None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:47:11.945065 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-48687ccc-9c1f-4c59-aead-caf9ea0826fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:47:12.010630 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 540/1086] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 1 14:47:11 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 14:47:12.276235 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5098b54e-0710-4587-a006-2485aecefdc6 None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:47:12.279118 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:47:12.376662 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5098b54e-0710-4587-a006-2485aecefdc6 None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:47:12.410013 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e65629f-846f-4cc7-9ec2-d7c4aa7c3888 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:47:12.410662 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e65629f-846f-4cc7-9ec2-d7c4aa7c3888 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:47:12.410943 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e65629f-846f-4cc7-9ec2-d7c4aa7c3888 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:47:12.411217 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e65629f-846f-4cc7-9ec2-d7c4aa7c3888 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:47:12.420938 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e65629f-846f-4cc7-9ec2-d7c4aa7c3888 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 14:47:12.425747 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application [None req-5e65629f-846f-4cc7-9ec2-d7c4aa7c3888 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 01 14:47:12.425747 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:47:12.425747 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:47:12.425747 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:47:12.425747 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:47:12.425747 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:47:12.425747 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:47:12.425747 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:47:12.425747 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:47:12.425747 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:47:12.425747 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:47:12.425747 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:47:12.425747 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 14:47:12.425747 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 14:47:12.425747 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 14:47:12.425747 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 14:47:12.426729 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 14:47:12.426729 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 14:47:12.426729 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 14:47:12.426729 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 14:47:12.426729 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:47:12.426729 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:47:12.426729 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:47:12.426729 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:47:12.426729 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:47:12.426729 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:47:12.426729 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:47:12.426729 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application with Lock( Sep 01 14:47:12.426729 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:47:12.426729 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:47:12.426729 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:47:12.426729 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:47:12.427884 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:47:12.427884 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:47:12.427884 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:47:12.427884 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:47:12.427884 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 14:47:12.427884 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 14:47:12.427884 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 14:47:12.427884 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 14:47:12.427884 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 14:47:12.427884 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 14:47:12.427884 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 01 14:47:12.427884 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Sep 01 14:47:12.427884 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 547/1087] 10.222.0.33 () {56 vars in 1120 bytes} [Thu Sep 1 14:47:12 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:47:12.432774 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48687ccc-9c1f-4c59-aead-caf9ea0826fb None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:47:12.435838 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:47:12.531464 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48687ccc-9c1f-4c59-aead-caf9ea0826fb None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:47:12.564383 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a7e023-da84-4527-bd64-340a917d822e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:47:12.564983 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a7e023-da84-4527-bd64-340a917d822e None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:47:12.565261 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a7e023-da84-4527-bd64-340a917d822e None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:47:12.566056 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a7e023-da84-4527-bd64-340a917d822e None admin] PATH_INFO: `/v3/roles` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:47:12.566754 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a7e023-da84-4527-bd64-340a917d822e None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81341) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:47:12.567109 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a7e023-da84-4527-bd64-340a917d822e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:47:12.568485 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a7e023-da84-4527-bd64-340a917d822e None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:47:12.576382 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 541/1088] 10.222.0.33 () {56 vars in 1115 bytes} [Thu Sep 1 14:47:12 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:47:12.582710 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e65629f-846f-4cc7-9ec2-d7c4aa7c3888 None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:47:12.655918 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:47:12.688257 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e65629f-846f-4cc7-9ec2-d7c4aa7c3888 None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:47:12.722738 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af3a8c5-7784-44a2-ad18-009e320256f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:47:12.723387 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af3a8c5-7784-44a2-ad18-009e320256f5 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:47:12.723661 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af3a8c5-7784-44a2-ad18-009e320256f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:47:12.723927 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af3a8c5-7784-44a2-ad18-009e320256f5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:47:12.734906 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af3a8c5-7784-44a2-ad18-009e320256f5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:47:12.736351 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af3a8c5-7784-44a2-ad18-009e320256f5 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:47:12.751081 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application [None req-3af3a8c5-7784-44a2-ad18-009e320256f5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 14:47:12.751081 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:47:12.751081 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:47:12.751081 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:47:12.751081 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:47:12.751081 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:47:12.751081 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:47:12.751081 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:47:12.751081 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:47:12.751081 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:47:12.751081 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:47:12.751081 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:47:12.751081 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 14:47:12.751081 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 14:47:12.751081 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 14:47:12.751081 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 14:47:12.752232 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:47:12.752232 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:47:12.752232 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:47:12.752232 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:47:12.752232 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:47:12.752232 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:47:12.752232 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:47:12.752232 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application with Lock( Sep 01 14:47:12.752232 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:47:12.752232 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:47:12.752232 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:47:12.752232 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:47:12.752232 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:47:12.752232 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:47:12.752232 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:47:12.752232 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:47:12.752232 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 14:47:12.753863 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 14:47:12.753863 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 14:47:12.753863 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Sep 01 14:47:12.753863 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 548/1089] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 1 14:47:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:47:12.758566 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a7e023-da84-4527-bd64-340a917d822e None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:47:12.822362 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:47:12.855961 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a7e023-da84-4527-bd64-340a917d822e None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:47:12.888241 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36255b29-e5ff-4de2-8821-3e99af951b9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:47:12.888833 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36255b29-e5ff-4de2-8821-3e99af951b9d None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:47:12.889106 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36255b29-e5ff-4de2-8821-3e99af951b9d None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:47:12.889373 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36255b29-e5ff-4de2-8821-3e99af951b9d None admin] PATH_INFO: `/v3/domains` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:47:12.890002 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36255b29-e5ff-4de2-8821-3e99af951b9d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81341) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:47:12.890330 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36255b29-e5ff-4de2-8821-3e99af951b9d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:47:12.891694 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36255b29-e5ff-4de2-8821-3e99af951b9d None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:47:12.902851 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 542/1090] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Sep 1 14:47:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:47:12.908515 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af3a8c5-7784-44a2-ad18-009e320256f5 None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:47:13.005602 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af3a8c5-7784-44a2-ad18-009e320256f5 None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:47:13.018656 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:47:13.038731 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334656c4-ce4f-48b7-9b9b-f76b57324daa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:47:13.039357 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334656c4-ce4f-48b7-9b9b-f76b57324daa None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:47:13.039636 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334656c4-ce4f-48b7-9b9b-f76b57324daa None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:47:13.039906 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334656c4-ce4f-48b7-9b9b-f76b57324daa None admin] PATH_INFO: `/v3/users/nova` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:47:13.047176 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334656c4-ce4f-48b7-9b9b-f76b57324daa None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:47:13.048607 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334656c4-ce4f-48b7-9b9b-f76b57324daa None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:47:13.059942 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application [None req-334656c4-ce4f-48b7-9b9b-f76b57324daa None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 01 14:47:13.059942 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:47:13.059942 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:47:13.059942 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:47:13.059942 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:47:13.059942 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:47:13.059942 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:47:13.059942 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:47:13.059942 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:47:13.059942 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:47:13.059942 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:47:13.059942 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:47:13.059942 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 14:47:13.059942 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 14:47:13.059942 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 14:47:13.059942 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 14:47:13.061061 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:47:13.061061 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:47:13.061061 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 14:47:13.061061 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 14:47:13.061061 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 14:47:13.061061 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 14:47:13.061061 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:47:13.061061 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:47:13.061061 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:47:13.061061 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:47:13.061061 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:47:13.061061 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application with Lock( Sep 01 14:47:13.061061 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:47:13.061061 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:47:13.061061 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:47:13.061061 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:47:13.061061 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:47:13.061061 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:47:13.062080 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:47:13.062080 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:47:13.062080 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 14:47:13.062080 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 14:47:13.062080 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 14:47:13.062080 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 14:47:13.062080 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 14:47:13.062080 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 14:47:13.062080 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 01 14:47:13.062080 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Sep 01 14:47:13.062080 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 549/1091] 10.222.0.33 () {56 vars in 1093 bytes} [Thu Sep 1 14:47:12 2022] GET /identity/v3/users/nova => generated 82 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:47:13.067497 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36255b29-e5ff-4de2-8821-3e99af951b9d None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:47:13.163782 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36255b29-e5ff-4de2-8821-3e99af951b9d None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:47:13.188720 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:47:13.197872 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7fbce3-1302-40fe-934e-038789ad6d59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:47:13.198503 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7fbce3-1302-40fe-934e-038789ad6d59 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:47:13.198784 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7fbce3-1302-40fe-934e-038789ad6d59 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:47:13.199055 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7fbce3-1302-40fe-934e-038789ad6d59 None admin] PATH_INFO: `/v3/users` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:47:13.199781 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7fbce3-1302-40fe-934e-038789ad6d59 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81341) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:47:13.200083 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7fbce3-1302-40fe-934e-038789ad6d59 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:47:13.201528 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7fbce3-1302-40fe-934e-038789ad6d59 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:47:13.218348 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 543/1092] 10.222.0.33 () {56 vars in 1133 bytes} [Thu Sep 1 14:47:13 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:47:13.224603 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334656c4-ce4f-48b7-9b9b-f76b57324daa None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:47:13.322787 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334656c4-ce4f-48b7-9b9b-f76b57324daa None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:47:13.355540 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85486064-3213-4cef-a73c-255c36b2668f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:47:13.356140 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85486064-3213-4cef-a73c-255c36b2668f None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:47:13.356419 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85486064-3213-4cef-a73c-255c36b2668f None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:47:13.356690 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85486064-3213-4cef-a73c-255c36b2668f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:47:13.397291 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:47:13.422628 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85486064-3213-4cef-a73c-255c36b2668f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:47:13.423999 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85486064-3213-4cef-a73c-255c36b2668f None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:47:13.519867 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 550/1093] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Sep 1 14:47:13 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 14:47:13.525607 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7fbce3-1302-40fe-934e-038789ad6d59 None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:47:13.561473 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa12ae24-ac13-461a-898b-0725664d9a97 None None] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:47:13.561753 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa12ae24-ac13-461a-898b-0725664d9a97 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:47:13.562025 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa12ae24-ac13-461a-898b-0725664d9a97 None None] PATH_INFO: `/` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:47:13.563381 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 551/1094] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 1 14:47:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 14:47:13.568255 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773cd7d8-87cc-4322-98b4-9ee9009a6e50 None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:47:13.568533 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773cd7d8-87cc-4322-98b4-9ee9009a6e50 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:47:13.568801 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773cd7d8-87cc-4322-98b4-9ee9009a6e50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:47:13.587143 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-773cd7d8-87cc-4322-98b4-9ee9009a6e50 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:47:13.588324 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-773cd7d8-87cc-4322-98b4-9ee9009a6e50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:47:13.623235 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7fbce3-1302-40fe-934e-038789ad6d59 None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:47:13.654365 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 552/1095] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 1 14:47:13 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 14:47:13.658845 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055534f0-602c-4b8a-a3be-4ff155a2204f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:47:13.660150 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055534f0-602c-4b8a-a3be-4ff155a2204f None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:47:13.660426 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055534f0-602c-4b8a-a3be-4ff155a2204f None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:47:13.660699 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055534f0-602c-4b8a-a3be-4ff155a2204f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:47:13.675061 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055534f0-602c-4b8a-a3be-4ff155a2204f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:47:13.676461 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055534f0-602c-4b8a-a3be-4ff155a2204f None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:47:13.690614 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application [None req-055534f0-602c-4b8a-a3be-4ff155a2204f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 14:47:13.690614 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:47:13.690614 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:47:13.690614 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:47:13.690614 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:47:13.690614 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:47:13.690614 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:47:13.690614 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:47:13.690614 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:47:13.690614 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:47:13.690614 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:47:13.690614 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:47:13.690614 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 14:47:13.690614 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 14:47:13.690614 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 14:47:13.690614 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 14:47:13.691575 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:47:13.691575 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:47:13.691575 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:47:13.691575 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:47:13.691575 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:47:13.691575 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:47:13.691575 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:47:13.691575 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application with Lock( Sep 01 14:47:13.691575 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:47:13.691575 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:47:13.691575 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:47:13.691575 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:47:13.691575 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:47:13.691575 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:47:13.691575 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:47:13.691575 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:47:13.691575 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 14:47:13.692541 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 14:47:13.692541 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 14:47:13.692541 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Sep 01 14:47:13.692541 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 544/1096] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 1 14:47:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:47:13.697557 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773cd7d8-87cc-4322-98b4-9ee9009a6e50 None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:47:13.701001 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:47:13.793991 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773cd7d8-87cc-4322-98b4-9ee9009a6e50 None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:47:13.826402 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c46cc8b-5fc3-417b-a312-4c2c8e63adae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:47:13.826994 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c46cc8b-5fc3-417b-a312-4c2c8e63adae None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:47:13.827266 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c46cc8b-5fc3-417b-a312-4c2c8e63adae None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:47:13.827528 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c46cc8b-5fc3-417b-a312-4c2c8e63adae None admin] PATH_INFO: `/v3/domains` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:47:13.828157 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c46cc8b-5fc3-417b-a312-4c2c8e63adae None admin] RBAC: Adding query filter params (name=Default) {{(pid=81342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:47:13.828449 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c46cc8b-5fc3-417b-a312-4c2c8e63adae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:47:13.829792 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c46cc8b-5fc3-417b-a312-4c2c8e63adae None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:47:13.841069 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 553/1097] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Sep 1 14:47:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:47:13.846643 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055534f0-602c-4b8a-a3be-4ff155a2204f None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:47:13.882231 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:47:13.919161 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c46cc8b-5fc3-417b-a312-4c2c8e63adae None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:47:13.945873 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055534f0-602c-4b8a-a3be-4ff155a2204f None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:47:13.979855 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:47:13.980001 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c86f94-624c-47fc-81bf-c0f58b2b66a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:47:13.980534 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c86f94-624c-47fc-81bf-c0f58b2b66a3 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:47:13.980814 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c86f94-624c-47fc-81bf-c0f58b2b66a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:47:13.981084 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c86f94-624c-47fc-81bf-c0f58b2b66a3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:47:13.988081 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c86f94-624c-47fc-81bf-c0f58b2b66a3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:47:13.989527 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c86f94-624c-47fc-81bf-c0f58b2b66a3 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:47:14.001210 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application [None req-65c86f94-624c-47fc-81bf-c0f58b2b66a3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 14:47:14.001210 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:47:14.001210 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:47:14.001210 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:47:14.001210 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:47:14.001210 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:47:14.001210 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:47:14.001210 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:47:14.001210 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:47:14.001210 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:47:14.001210 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:47:14.001210 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:47:14.001210 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 14:47:14.001210 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 14:47:14.001210 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 14:47:14.007765 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 14:47:14.007765 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:47:14.007765 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:47:14.007765 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:47:14.007765 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:47:14.007765 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:47:14.007765 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:47:14.007765 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:47:14.007765 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application with Lock( Sep 01 14:47:14.007765 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:47:14.007765 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:47:14.007765 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:47:14.007765 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:47:14.007765 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:47:14.007765 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:47:14.007765 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:47:14.007765 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:47:14.008793 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 14:47:14.008793 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 14:47:14.008793 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 14:47:14.008793 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 14:47:14.008793 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 14:47:14.008793 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 14:47:14.008793 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 14:47:14.008793 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Sep 01 14:47:14.008793 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 545/1098] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Sep 1 14:47:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:47:14.008793 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c86f94-624c-47fc-81bf-c0f58b2b66a3 None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:47:14.018023 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c46cc8b-5fc3-417b-a312-4c2c8e63adae None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:47:14.051107 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745a16ca-5f8a-419a-b577-d0311c9cee7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:47:14.051720 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745a16ca-5f8a-419a-b577-d0311c9cee7a None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:47:14.051996 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745a16ca-5f8a-419a-b577-d0311c9cee7a None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:47:14.052265 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745a16ca-5f8a-419a-b577-d0311c9cee7a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:47:14.062777 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745a16ca-5f8a-419a-b577-d0311c9cee7a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 14:47:14.068022 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application [None req-745a16ca-5f8a-419a-b577-d0311c9cee7a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 14:47:14.068022 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:47:14.068022 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:47:14.068022 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:47:14.068022 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:47:14.068022 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:47:14.068022 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:47:14.068022 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:47:14.068022 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:47:14.068022 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:47:14.068022 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:47:14.068022 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:47:14.068022 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 14:47:14.068022 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 14:47:14.068022 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 14:47:14.068022 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 14:47:14.069183 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 14:47:14.069183 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 14:47:14.069183 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 14:47:14.069183 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 14:47:14.069183 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:47:14.069183 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:47:14.069183 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:47:14.069183 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:47:14.069183 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:47:14.069183 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:47:14.069183 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:47:14.069183 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application with Lock( Sep 01 14:47:14.069183 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:47:14.069183 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:47:14.069183 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:47:14.069183 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:47:14.070344 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:47:14.070344 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:47:14.070344 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:47:14.070344 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:47:14.070344 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 14:47:14.070344 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 14:47:14.070344 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 14:47:14.070344 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 14:47:14.070344 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 14:47:14.070344 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 14:47:14.070344 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 14:47:14.070344 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Sep 01 14:47:14.070344 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 554/1099] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Sep 1 14:47:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 14:47:14.075170 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745a16ca-5f8a-419a-b577-d0311c9cee7a None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:47:14.099958 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:47:14.106068 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c86f94-624c-47fc-81bf-c0f58b2b66a3 None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:47:14.138137 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f057da6-881d-4571-bc42-703cf89b7cc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:47:14.138758 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f057da6-881d-4571-bc42-703cf89b7cc6 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:47:14.139040 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f057da6-881d-4571-bc42-703cf89b7cc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:47:14.139568 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f057da6-881d-4571-bc42-703cf89b7cc6 None admin] PATH_INFO: `/v3/projects` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:47:14.140243 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f057da6-881d-4571-bc42-703cf89b7cc6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81341) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:47:14.140543 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f057da6-881d-4571-bc42-703cf89b7cc6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:47:14.141964 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f057da6-881d-4571-bc42-703cf89b7cc6 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:47:14.154218 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 546/1100] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Sep 1 14:47:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 14:47:14.160227 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f057da6-881d-4571-bc42-703cf89b7cc6 None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:47:14.172177 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745a16ca-5f8a-419a-b577-d0311c9cee7a None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:47:14.195234 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:47:14.206729 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a7bba1-e33d-416a-9244-ea2747564fae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:47:14.207285 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a7bba1-e33d-416a-9244-ea2747564fae None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:47:14.207558 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a7bba1-e33d-416a-9244-ea2747564fae None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:47:14.207825 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a7bba1-e33d-416a-9244-ea2747564fae None admin] PATH_INFO: `/v3/roles` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:47:14.208484 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a7bba1-e33d-416a-9244-ea2747564fae None admin] RBAC: Adding query filter params (name=admin) {{(pid=81342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:47:14.208787 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a7bba1-e33d-416a-9244-ea2747564fae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:47:14.210139 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a7bba1-e33d-416a-9244-ea2747564fae None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:47:14.218401 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 555/1101] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Sep 1 14:47:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 14:47:14.224134 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a7bba1-e33d-416a-9244-ea2747564fae None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:47:14.257976 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f057da6-881d-4571-bc42-703cf89b7cc6 None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:47:14.291150 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e84dff-874f-462c-85de-2107a5ed712c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:47:14.291737 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e84dff-874f-462c-85de-2107a5ed712c None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:47:14.292011 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e84dff-874f-462c-85de-2107a5ed712c None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:47:14.292276 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e84dff-874f-462c-85de-2107a5ed712c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:47:14.292989 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e84dff-874f-462c-85de-2107a5ed712c None admin] RBAC: Adding query filter params (role.id=050916e5d27c476c9eed7850b136dd98, scope.project.id=e8c85b4bd17e4e89b56bf0f831b474d5, user.id=4858bb8621854ec8aba836e7d9050498) {{(pid=81341) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:47:14.293288 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e84dff-874f-462c-85de-2107a5ed712c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:47:14.294750 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e84dff-874f-462c-85de-2107a5ed712c None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:47:14.300771 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 547/1102] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Sep 1 14:47:14 2022] GET /identity/v3/role_assignments?user.id=4858bb8621854ec8aba836e7d9050498&scope.project.id=e8c85b4bd17e4e89b56bf0f831b474d5&role.id=050916e5d27c476c9eed7850b136dd98 => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 14:47:14.323999 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a7bba1-e33d-416a-9244-ea2747564fae None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:47:14.360379 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be78cb95-f86c-4d91-a810-1c8263407072 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:47:14.361023 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be78cb95-f86c-4d91-a810-1c8263407072 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:47:14.361299 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be78cb95-f86c-4d91-a810-1c8263407072 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:47:14.361567 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be78cb95-f86c-4d91-a810-1c8263407072 None admin] PATH_INFO: `/v3/users/e7203c765cb446eabeb1d04cfc2e9f89` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:47:14.363315 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be78cb95-f86c-4d91-a810-1c8263407072 None admin] RBAC: Authorizing `identity:get_user(user_id=e7203c765cb446eabeb1d04cfc2e9f89)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:47:14.364774 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be78cb95-f86c-4d91-a810-1c8263407072 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:47:14.366469 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 556/1103] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Sep 1 14:47:14 2022] GET /identity/v3/users/e7203c765cb446eabeb1d04cfc2e9f89 => generated 285 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:47:14.372564 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e84dff-874f-462c-85de-2107a5ed712c None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:47:14.396044 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:47:14.476653 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e84dff-874f-462c-85de-2107a5ed712c None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:47:14.510724 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c9e39f-251a-494e-97a7-fb322f13c77e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:47:14.511330 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c9e39f-251a-494e-97a7-fb322f13c77e None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:47:14.511602 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c9e39f-251a-494e-97a7-fb322f13c77e None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:47:14.511867 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c9e39f-251a-494e-97a7-fb322f13c77e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:47:14.578461 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c9e39f-251a-494e-97a7-fb322f13c77e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:47:14.580061 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c9e39f-251a-494e-97a7-fb322f13c77e None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:47:14.601121 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:47:14.682046 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 548/1104] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Sep 1 14:47:14 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 01 14:47:14.689087 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be78cb95-f86c-4d91-a810-1c8263407072 None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:47:14.745950 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:47:14.795205 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be78cb95-f86c-4d91-a810-1c8263407072 None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:47:14.834004 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2561db03-dfbd-45d6-b7d8-f8dffd6e4659 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:47:14.834664 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2561db03-dfbd-45d6-b7d8-f8dffd6e4659 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:47:14.834948 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2561db03-dfbd-45d6-b7d8-f8dffd6e4659 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:47:14.835220 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2561db03-dfbd-45d6-b7d8-f8dffd6e4659 None admin] PATH_INFO: `/v3/projects/1f8ecb764c6a437bb0911fe60a148b9c` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:47:14.836337 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2561db03-dfbd-45d6-b7d8-f8dffd6e4659 None admin] RBAC: Authorizing `identity:get_project(project_id=1f8ecb764c6a437bb0911fe60a148b9c)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:47:14.837756 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2561db03-dfbd-45d6-b7d8-f8dffd6e4659 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:47:14.839423 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 557/1105] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Sep 1 14:47:14 2022] GET /identity/v3/projects/1f8ecb764c6a437bb0911fe60a148b9c => generated 311 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:47:14.845603 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c9e39f-251a-494e-97a7-fb322f13c77e None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:47:14.949395 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c9e39f-251a-494e-97a7-fb322f13c77e None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:47:14.970369 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:47:14.985785 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a81886-651a-4bda-9819-3c1fdf35867b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:47:14.986475 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a81886-651a-4bda-9819-3c1fdf35867b None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:47:14.986775 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a81886-651a-4bda-9819-3c1fdf35867b None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:47:14.987024 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a81886-651a-4bda-9819-3c1fdf35867b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:47:14.987782 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a81886-651a-4bda-9819-3c1fdf35867b None admin] RBAC: Adding query filter params (role.id=2a2adb769e55476b82e5014897e0a9c0, scope.project.id=1f8ecb764c6a437bb0911fe60a148b9c, user.id=e7203c765cb446eabeb1d04cfc2e9f89) {{(pid=81341) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:47:14.988088 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a81886-651a-4bda-9819-3c1fdf35867b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:47:14.989525 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a81886-651a-4bda-9819-3c1fdf35867b None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:47:14.995349 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 549/1106] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Sep 1 14:47:14 2022] GET /identity/v3/role_assignments?user.id=e7203c765cb446eabeb1d04cfc2e9f89&scope.project.id=1f8ecb764c6a437bb0911fe60a148b9c&role.id=2a2adb769e55476b82e5014897e0a9c0 => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:47:16.906622 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12af7efa-f61b-4697-a66f-55715922f17e None None] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:47:16.907143 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12af7efa-f61b-4697-a66f-55715922f17e None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:47:16.907274 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12af7efa-f61b-4697-a66f-55715922f17e None None] PATH_INFO: `/` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:47:16.908492 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 558/1107] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 1 14:47:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:47:16.914104 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872353ef-19e6-4d36-b599-f15ed8eba156 None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:47:16.914539 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872353ef-19e6-4d36-b599-f15ed8eba156 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:47:16.914648 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872353ef-19e6-4d36-b599-f15ed8eba156 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:47:16.935947 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-872353ef-19e6-4d36-b599-f15ed8eba156 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:47:16.937162 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-872353ef-19e6-4d36-b599-f15ed8eba156 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:47:17.001829 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 550/1108] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 1 14:47:16 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 14:47:17.270141 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12af7efa-f61b-4697-a66f-55715922f17e None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:47:17.368771 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12af7efa-f61b-4697-a66f-55715922f17e None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:47:17.389795 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:47:17.401664 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b610d5-2dd5-4dc5-84fa-c0619f3e06f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:47:17.402297 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b610d5-2dd5-4dc5-84fa-c0619f3e06f1 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:47:17.402572 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b610d5-2dd5-4dc5-84fa-c0619f3e06f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:47:17.402836 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b610d5-2dd5-4dc5-84fa-c0619f3e06f1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:47:17.403918 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b610d5-2dd5-4dc5-84fa-c0619f3e06f1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:47:17.405331 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b610d5-2dd5-4dc5-84fa-c0619f3e06f1 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:47:17.406895 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 559/1109] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 1 14:47:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:47:17.413145 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872353ef-19e6-4d36-b599-f15ed8eba156 None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:47:17.507536 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872353ef-19e6-4d36-b599-f15ed8eba156 None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:47:17.526962 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:47:17.546857 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d93b88-f9f3-4353-804a-201b89b37c17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:47:17.547449 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d93b88-f9f3-4353-804a-201b89b37c17 None admin] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:47:17.547721 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d93b88-f9f3-4353-804a-201b89b37c17 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:47:17.547987 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d93b88-f9f3-4353-804a-201b89b37c17 None admin] PATH_INFO: `/v3/users` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:47:17.548629 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d93b88-f9f3-4353-804a-201b89b37c17 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:47:17.550064 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d93b88-f9f3-4353-804a-201b89b37c17 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:47:17.570949 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 551/1110] 10.222.0.33 () {60 vars in 1132 bytes} [Thu Sep 1 14:47:17 2022] POST /identity/v3/users => generated 286 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:47:19.463806 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69cef69-053c-4f52-82c8-32902aedde3f None None] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:47:19.464446 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69cef69-053c-4f52-82c8-32902aedde3f None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:47:19.464446 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69cef69-053c-4f52-82c8-32902aedde3f None None] PATH_INFO: `/` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:47:19.465656 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 560/1111] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 1 14:47:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:47:19.470904 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbacbfbe-8fb1-4a7e-a1d2-8abe6679ed4b None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:47:19.471334 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbacbfbe-8fb1-4a7e-a1d2-8abe6679ed4b None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:47:19.471439 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbacbfbe-8fb1-4a7e-a1d2-8abe6679ed4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:47:19.489540 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-cbacbfbe-8fb1-4a7e-a1d2-8abe6679ed4b None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:47:19.490762 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-cbacbfbe-8fb1-4a7e-a1d2-8abe6679ed4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:47:19.555206 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 552/1112] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 1 14:47:19 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 14:47:19.823134 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69cef69-053c-4f52-82c8-32902aedde3f None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:47:19.919083 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69cef69-053c-4f52-82c8-32902aedde3f None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:47:19.951604 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d0b911-b400-4915-aac7-964b618b5bd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:47:19.952206 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d0b911-b400-4915-aac7-964b618b5bd5 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:47:19.952480 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d0b911-b400-4915-aac7-964b618b5bd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:47:19.952745 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d0b911-b400-4915-aac7-964b618b5bd5 None admin] PATH_INFO: `/v3/roles/9996407d588d4a00836c8c50e7f37b1a` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:47:19.954101 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d0b911-b400-4915-aac7-964b618b5bd5 None admin] RBAC: Authorizing `identity:get_role(role_id=9996407d588d4a00836c8c50e7f37b1a)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:47:19.955471 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d0b911-b400-4915-aac7-964b618b5bd5 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:47:19.956765 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 561/1113] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Sep 1 14:47:19 2022] GET /identity/v3/roles/9996407d588d4a00836c8c50e7f37b1a => generated 223 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:47:19.962424 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbacbfbe-8fb1-4a7e-a1d2-8abe6679ed4b None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:47:20.055714 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:47:20.058721 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbacbfbe-8fb1-4a7e-a1d2-8abe6679ed4b None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:47:20.091420 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a0b6d8-a7fe-4e1a-972b-988b3e6bc3fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:47:20.091976 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a0b6d8-a7fe-4e1a-972b-988b3e6bc3fc None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:47:20.092247 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a0b6d8-a7fe-4e1a-972b-988b3e6bc3fc None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:47:20.092514 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a0b6d8-a7fe-4e1a-972b-988b3e6bc3fc None admin] PATH_INFO: `/v3/users/7ecfc63d99a840dcb61855a556f935e8` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:47:20.105358 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a0b6d8-a7fe-4e1a-972b-988b3e6bc3fc None admin] RBAC: Authorizing `identity:get_user(user_id=7ecfc63d99a840dcb61855a556f935e8)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:47:20.106838 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a0b6d8-a7fe-4e1a-972b-988b3e6bc3fc None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:47:20.108401 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 553/1114] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Sep 1 14:47:19 2022] GET /identity/v3/users/7ecfc63d99a840dcb61855a556f935e8 => generated 286 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:47:20.114964 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d0b911-b400-4915-aac7-964b618b5bd5 None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:47:20.171534 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:47:20.213595 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d0b911-b400-4915-aac7-964b618b5bd5 None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:47:20.245715 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442abeee-9487-4e9f-acdc-d7235caa5872 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:47:20.246323 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442abeee-9487-4e9f-acdc-d7235caa5872 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:47:20.246604 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442abeee-9487-4e9f-acdc-d7235caa5872 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:47:20.246871 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442abeee-9487-4e9f-acdc-d7235caa5872 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:47:20.310698 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442abeee-9487-4e9f-acdc-d7235caa5872 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:47:20.312133 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442abeee-9487-4e9f-acdc-d7235caa5872 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:47:20.366383 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:47:20.406969 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 562/1115] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Sep 1 14:47:20 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 14:47:20.413360 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a0b6d8-a7fe-4e1a-972b-988b3e6bc3fc None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:47:20.507320 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a0b6d8-a7fe-4e1a-972b-988b3e6bc3fc None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:47:20.540520 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05d4e3d-589d-44df-ac3c-491666967c2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:47:20.541122 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05d4e3d-589d-44df-ac3c-491666967c2b None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:47:20.541398 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05d4e3d-589d-44df-ac3c-491666967c2b None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:47:20.541663 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05d4e3d-589d-44df-ac3c-491666967c2b None admin] PATH_INFO: `/v3/projects/1f8ecb764c6a437bb0911fe60a148b9c` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:47:20.542779 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05d4e3d-589d-44df-ac3c-491666967c2b None admin] RBAC: Authorizing `identity:get_project(project_id=1f8ecb764c6a437bb0911fe60a148b9c)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:47:20.544202 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05d4e3d-589d-44df-ac3c-491666967c2b None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:47:20.545772 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 554/1116] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Sep 1 14:47:20 2022] GET /identity/v3/projects/1f8ecb764c6a437bb0911fe60a148b9c => generated 311 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:47:20.551715 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442abeee-9487-4e9f-acdc-d7235caa5872 None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:47:20.647414 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442abeee-9487-4e9f-acdc-d7235caa5872 None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:47:20.680420 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d60206d-e434-4490-8075-56de571f1f49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:47:20.681016 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d60206d-e434-4490-8075-56de571f1f49 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:47:20.681293 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d60206d-e434-4490-8075-56de571f1f49 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:47:20.681561 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d60206d-e434-4490-8075-56de571f1f49 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:47:20.682308 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d60206d-e434-4490-8075-56de571f1f49 None admin] RBAC: Adding query filter params (role.id=9996407d588d4a00836c8c50e7f37b1a, scope.project.id=1f8ecb764c6a437bb0911fe60a148b9c, user.id=7ecfc63d99a840dcb61855a556f935e8) {{(pid=81342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:47:20.682616 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d60206d-e434-4490-8075-56de571f1f49 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:47:20.684025 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d60206d-e434-4490-8075-56de571f1f49 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:47:20.689210 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 563/1117] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Sep 1 14:47:20 2022] GET /identity/v3/role_assignments?user.id=7ecfc63d99a840dcb61855a556f935e8&scope.project.id=1f8ecb764c6a437bb0911fe60a148b9c&role.id=9996407d588d4a00836c8c50e7f37b1a => generated 259 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:47:22.530713 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ada73b-f3c1-4426-94dc-cfe9366e0825 None None] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:47:22.531307 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ada73b-f3c1-4426-94dc-cfe9366e0825 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:47:22.531307 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ada73b-f3c1-4426-94dc-cfe9366e0825 None None] PATH_INFO: `/` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:47:22.533544 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 555/1118] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 1 14:47:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:47:22.538917 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7510fdcc-e578-41b9-b540-76616c7e3f96 None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:47:22.539318 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7510fdcc-e578-41b9-b540-76616c7e3f96 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:47:22.539466 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7510fdcc-e578-41b9-b540-76616c7e3f96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:47:22.558252 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-7510fdcc-e578-41b9-b540-76616c7e3f96 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:47:22.559444 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-7510fdcc-e578-41b9-b540-76616c7e3f96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:47:22.576396 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:47:22.623976 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 564/1119] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 1 14:47:22 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 14:47:22.894966 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ada73b-f3c1-4426-94dc-cfe9366e0825 None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:47:22.922101 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:47:22.994907 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ada73b-f3c1-4426-94dc-cfe9366e0825 None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:47:23.026986 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f819cd-6031-4062-9a5c-785249a709d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:47:23.027594 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f819cd-6031-4062-9a5c-785249a709d3 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:47:23.027867 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f819cd-6031-4062-9a5c-785249a709d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:47:23.028133 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f819cd-6031-4062-9a5c-785249a709d3 None admin] PATH_INFO: `/v3/roles/9996407d588d4a00836c8c50e7f37b1a` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:47:23.029496 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f819cd-6031-4062-9a5c-785249a709d3 None admin] RBAC: Authorizing `identity:get_role(role_id=9996407d588d4a00836c8c50e7f37b1a)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:47:23.030891 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f819cd-6031-4062-9a5c-785249a709d3 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:47:23.032178 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 556/1120] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Sep 1 14:47:22 2022] GET /identity/v3/roles/9996407d588d4a00836c8c50e7f37b1a => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:47:23.038424 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7510fdcc-e578-41b9-b540-76616c7e3f96 None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:47:23.132757 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7510fdcc-e578-41b9-b540-76616c7e3f96 None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:47:23.164740 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0851d0-e146-4289-9950-4a88e2f88af6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:47:23.165285 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0851d0-e146-4289-9950-4a88e2f88af6 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:47:23.165557 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0851d0-e146-4289-9950-4a88e2f88af6 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:47:23.165824 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0851d0-e146-4289-9950-4a88e2f88af6 None admin] PATH_INFO: `/v3/users/7ecfc63d99a840dcb61855a556f935e8` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:47:23.166932 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0851d0-e146-4289-9950-4a88e2f88af6 None admin] RBAC: Authorizing `identity:get_user(user_id=7ecfc63d99a840dcb61855a556f935e8)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:47:23.168337 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0851d0-e146-4289-9950-4a88e2f88af6 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:47:23.169849 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 565/1121] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Sep 1 14:47:23 2022] GET /identity/v3/users/7ecfc63d99a840dcb61855a556f935e8 => generated 286 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:47:23.175361 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f819cd-6031-4062-9a5c-785249a709d3 None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:47:23.265246 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:47:23.271931 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f819cd-6031-4062-9a5c-785249a709d3 None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:47:23.304338 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50633aba-97cb-4a54-8318-d3eac0561ae3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:47:23.304950 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50633aba-97cb-4a54-8318-d3eac0561ae3 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:47:23.305227 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50633aba-97cb-4a54-8318-d3eac0561ae3 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:47:23.305495 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50633aba-97cb-4a54-8318-d3eac0561ae3 None admin] PATH_INFO: `/v3/projects/1f8ecb764c6a437bb0911fe60a148b9c` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:47:23.306629 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50633aba-97cb-4a54-8318-d3eac0561ae3 None admin] RBAC: Authorizing `identity:get_project(project_id=1f8ecb764c6a437bb0911fe60a148b9c)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:47:23.308107 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50633aba-97cb-4a54-8318-d3eac0561ae3 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:47:23.309683 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 557/1122] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Sep 1 14:47:23 2022] GET /identity/v3/projects/1f8ecb764c6a437bb0911fe60a148b9c => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:47:23.316166 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0851d0-e146-4289-9950-4a88e2f88af6 None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:47:23.337623 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:47:23.411803 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0851d0-e146-4289-9950-4a88e2f88af6 None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:47:23.453031 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af061313-d4a4-4957-ac03-d74ef996dbae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:47:23.453614 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af061313-d4a4-4957-ac03-d74ef996dbae None admin] REQUEST_METHOD: `PUT` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:47:23.453893 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af061313-d4a4-4957-ac03-d74ef996dbae None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:47:23.454164 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af061313-d4a4-4957-ac03-d74ef996dbae None admin] PATH_INFO: `/v3/projects/1f8ecb764c6a437bb0911fe60a148b9c/users/7ecfc63d99a840dcb61855a556f935e8/roles/9996407d588d4a00836c8c50e7f37b1a` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:47:23.456887 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af061313-d4a4-4957-ac03-d74ef996dbae None admin] RBAC: Authorizing `identity:create_grant(project_id=1f8ecb764c6a437bb0911fe60a148b9c, user_id=7ecfc63d99a840dcb61855a556f935e8, role_id=9996407d588d4a00836c8c50e7f37b1a)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:47:23.458424 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af061313-d4a4-4957-ac03-d74ef996dbae None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:47:23.471935 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 566/1123] 10.222.0.33 () {58 vars in 1439 bytes} [Thu Sep 1 14:47:23 2022] PUT /identity/v3/projects/1f8ecb764c6a437bb0911fe60a148b9c/users/7ecfc63d99a840dcb61855a556f935e8/roles/9996407d588d4a00836c8c50e7f37b1a => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:47:25.353445 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3af7279-cbdf-41ba-b696-c455d6a30059 None None] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:47:25.353973 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3af7279-cbdf-41ba-b696-c455d6a30059 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:47:25.353973 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3af7279-cbdf-41ba-b696-c455d6a30059 None None] PATH_INFO: `/` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:47:25.355382 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 558/1124] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 1 14:47:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:47:25.360384 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb676e59-ae13-4ac5-b81c-e33f5545ccda None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:47:25.360765 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb676e59-ae13-4ac5-b81c-e33f5545ccda None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:47:25.360930 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb676e59-ae13-4ac5-b81c-e33f5545ccda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:47:25.379137 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-eb676e59-ae13-4ac5-b81c-e33f5545ccda None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:47:25.380315 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-eb676e59-ae13-4ac5-b81c-e33f5545ccda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:47:25.443946 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 567/1125] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 1 14:47:25 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 14:47:25.711285 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3af7279-cbdf-41ba-b696-c455d6a30059 None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:47:25.809260 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3af7279-cbdf-41ba-b696-c455d6a30059 None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:47:25.842052 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab59475-cfee-435f-8772-a6015cfcbf49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:47:25.842688 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab59475-cfee-435f-8772-a6015cfcbf49 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:47:25.842970 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab59475-cfee-435f-8772-a6015cfcbf49 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:47:25.843242 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab59475-cfee-435f-8772-a6015cfcbf49 None admin] PATH_INFO: `/v3/roles/9996407d588d4a00836c8c50e7f37b1a` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:47:25.844625 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab59475-cfee-435f-8772-a6015cfcbf49 None admin] RBAC: Authorizing `identity:get_role(role_id=9996407d588d4a00836c8c50e7f37b1a)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:47:25.845995 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab59475-cfee-435f-8772-a6015cfcbf49 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:47:25.847828 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 559/1126] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Sep 1 14:47:25 2022] GET /identity/v3/roles/9996407d588d4a00836c8c50e7f37b1a => generated 223 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:47:25.853396 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb676e59-ae13-4ac5-b81c-e33f5545ccda None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:47:25.865223 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:47:25.949861 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb676e59-ae13-4ac5-b81c-e33f5545ccda None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:47:25.982102 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6361087-c95c-4f1a-bda8-7ee15bbc2c89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:47:25.982738 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6361087-c95c-4f1a-bda8-7ee15bbc2c89 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:47:25.983013 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6361087-c95c-4f1a-bda8-7ee15bbc2c89 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:47:25.983277 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6361087-c95c-4f1a-bda8-7ee15bbc2c89 None admin] PATH_INFO: `/v3/users/7ecfc63d99a840dcb61855a556f935e8` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:47:25.984368 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6361087-c95c-4f1a-bda8-7ee15bbc2c89 None admin] RBAC: Authorizing `identity:get_user(user_id=7ecfc63d99a840dcb61855a556f935e8)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:47:25.985774 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6361087-c95c-4f1a-bda8-7ee15bbc2c89 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:47:25.988082 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 568/1127] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Sep 1 14:47:25 2022] GET /identity/v3/users/7ecfc63d99a840dcb61855a556f935e8 => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 14:47:25.993564 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab59475-cfee-435f-8772-a6015cfcbf49 None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:47:26.022508 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:47:26.088612 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab59475-cfee-435f-8772-a6015cfcbf49 None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:47:26.121547 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f982d36a-0575-49f1-a6d4-1bcd3b1f93d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:47:26.122126 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f982d36a-0575-49f1-a6d4-1bcd3b1f93d8 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:47:26.122432 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f982d36a-0575-49f1-a6d4-1bcd3b1f93d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:47:26.122699 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f982d36a-0575-49f1-a6d4-1bcd3b1f93d8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:47:26.185992 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f982d36a-0575-49f1-a6d4-1bcd3b1f93d8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:47:26.187464 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f982d36a-0575-49f1-a6d4-1bcd3b1f93d8 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:47:26.215205 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:47:26.282009 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 560/1128] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Sep 1 14:47:25 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 14:47:26.288539 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6361087-c95c-4f1a-bda8-7ee15bbc2c89 None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:47:26.363765 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:47:26.384551 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6361087-c95c-4f1a-bda8-7ee15bbc2c89 None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:47:26.417688 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a186180f-0155-4b4f-8b34-af5b64b8d570 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:47:26.418313 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a186180f-0155-4b4f-8b34-af5b64b8d570 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:47:26.418593 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a186180f-0155-4b4f-8b34-af5b64b8d570 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:47:26.418860 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a186180f-0155-4b4f-8b34-af5b64b8d570 None admin] PATH_INFO: `/v3/projects/1f8ecb764c6a437bb0911fe60a148b9c` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:47:26.419951 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a186180f-0155-4b4f-8b34-af5b64b8d570 None admin] RBAC: Authorizing `identity:get_project(project_id=1f8ecb764c6a437bb0911fe60a148b9c)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:47:26.421351 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a186180f-0155-4b4f-8b34-af5b64b8d570 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:47:26.422969 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 569/1129] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Sep 1 14:47:26 2022] GET /identity/v3/projects/1f8ecb764c6a437bb0911fe60a148b9c => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:47:26.428909 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f982d36a-0575-49f1-a6d4-1bcd3b1f93d8 None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:47:26.524609 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f982d36a-0575-49f1-a6d4-1bcd3b1f93d8 None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:47:26.552756 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:47:26.558776 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555add13-eaa3-40b8-8b20-1a7b6066441a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:47:26.559363 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555add13-eaa3-40b8-8b20-1a7b6066441a None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:47:26.559639 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555add13-eaa3-40b8-8b20-1a7b6066441a None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:47:26.559905 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555add13-eaa3-40b8-8b20-1a7b6066441a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:47:26.560613 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555add13-eaa3-40b8-8b20-1a7b6066441a None admin] RBAC: Adding query filter params (role.id=9996407d588d4a00836c8c50e7f37b1a, scope.project.id=1f8ecb764c6a437bb0911fe60a148b9c, user.id=7ecfc63d99a840dcb61855a556f935e8) {{(pid=81341) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:47:26.560917 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555add13-eaa3-40b8-8b20-1a7b6066441a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:47:26.562503 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555add13-eaa3-40b8-8b20-1a7b6066441a None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:47:26.569185 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 561/1130] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Sep 1 14:47:26 2022] GET /identity/v3/role_assignments?user.id=7ecfc63d99a840dcb61855a556f935e8&scope.project.id=1f8ecb764c6a437bb0911fe60a148b9c&role.id=9996407d588d4a00836c8c50e7f37b1a => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:47:28.469641 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505ab6a3-d25c-41bf-8147-426dd2d41f8f None None] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:47:28.470183 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505ab6a3-d25c-41bf-8147-426dd2d41f8f None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:47:28.470183 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505ab6a3-d25c-41bf-8147-426dd2d41f8f None None] PATH_INFO: `/` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:47:28.471529 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 570/1131] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 1 14:47:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:47:28.476552 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e468cc33-155c-4594-b628-af2b81d2d718 None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:47:28.476915 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e468cc33-155c-4594-b628-af2b81d2d718 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:47:28.477095 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e468cc33-155c-4594-b628-af2b81d2d718 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:47:28.495251 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-e468cc33-155c-4594-b628-af2b81d2d718 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:47:28.496454 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-e468cc33-155c-4594-b628-af2b81d2d718 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:47:28.559971 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 562/1132] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 1 14:47:28 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 14:47:28.827320 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505ab6a3-d25c-41bf-8147-426dd2d41f8f None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:47:28.926212 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505ab6a3-d25c-41bf-8147-426dd2d41f8f None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:47:28.958308 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e224c3-89d2-476e-b4d7-4c5c4850d029 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:47:28.958906 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e224c3-89d2-476e-b4d7-4c5c4850d029 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:47:28.959177 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e224c3-89d2-476e-b4d7-4c5c4850d029 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:47:28.959438 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e224c3-89d2-476e-b4d7-4c5c4850d029 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:47:28.960520 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e224c3-89d2-476e-b4d7-4c5c4850d029 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:47:28.961926 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e224c3-89d2-476e-b4d7-4c5c4850d029 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:47:28.963472 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 571/1133] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 1 14:47:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:47:28.969211 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e468cc33-155c-4594-b628-af2b81d2d718 None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:47:29.066577 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e468cc33-155c-4594-b628-af2b81d2d718 None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:47:29.088922 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:47:29.100503 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7453ff-c698-4489-82c8-a50a004481ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:47:29.101133 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7453ff-c698-4489-82c8-a50a004481ca None admin] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:47:29.101411 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7453ff-c698-4489-82c8-a50a004481ca None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:47:29.101681 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7453ff-c698-4489-82c8-a50a004481ca None admin] PATH_INFO: `/v3/projects` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:47:29.102370 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7453ff-c698-4489-82c8-a50a004481ca None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:47:29.103809 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7453ff-c698-4489-82c8-a50a004481ca None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:47:29.132871 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 563/1134] 10.222.0.33 () {60 vars in 1141 bytes} [Thu Sep 1 14:47:28 2022] POST /identity/v3/projects => generated 311 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:47:31.031261 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ab7de9-e5a7-49cb-86c7-d4beaa97d000 None None] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:47:31.031765 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ab7de9-e5a7-49cb-86c7-d4beaa97d000 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:47:31.031765 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ab7de9-e5a7-49cb-86c7-d4beaa97d000 None None] PATH_INFO: `/` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:47:31.033069 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 572/1135] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 1 14:47:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:47:31.038207 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c79d04f-00f3-4909-9529-1813d5ba8672 None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:47:31.038641 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c79d04f-00f3-4909-9529-1813d5ba8672 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:47:31.038778 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c79d04f-00f3-4909-9529-1813d5ba8672 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:47:31.056922 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-4c79d04f-00f3-4909-9529-1813d5ba8672 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:47:31.058126 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-4c79d04f-00f3-4909-9529-1813d5ba8672 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:47:31.122141 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 564/1136] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 1 14:47:31 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 14:47:31.388357 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ab7de9-e5a7-49cb-86c7-d4beaa97d000 None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:47:31.403730 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:47:31.488510 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ab7de9-e5a7-49cb-86c7-d4beaa97d000 None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:47:31.521531 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d944a02-71e6-4cae-b124-23b1fd219fc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:47:31.522143 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d944a02-71e6-4cae-b124-23b1fd219fc7 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:47:31.522446 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d944a02-71e6-4cae-b124-23b1fd219fc7 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:47:31.522713 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d944a02-71e6-4cae-b124-23b1fd219fc7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:47:31.523807 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d944a02-71e6-4cae-b124-23b1fd219fc7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:47:31.525219 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d944a02-71e6-4cae-b124-23b1fd219fc7 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:47:31.526804 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 573/1137] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 1 14:47:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:47:31.532541 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c79d04f-00f3-4909-9529-1813d5ba8672 None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:47:31.626019 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c79d04f-00f3-4909-9529-1813d5ba8672 None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:47:31.632935 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:47:31.660465 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2d4af7-93a6-4ab0-b006-bf4221df3eb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:47:31.661001 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2d4af7-93a6-4ab0-b006-bf4221df3eb5 None admin] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:47:31.661276 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2d4af7-93a6-4ab0-b006-bf4221df3eb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:47:31.661543 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2d4af7-93a6-4ab0-b006-bf4221df3eb5 None admin] PATH_INFO: `/v3/users` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:47:31.662191 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2d4af7-93a6-4ab0-b006-bf4221df3eb5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:47:31.663703 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2d4af7-93a6-4ab0-b006-bf4221df3eb5 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:47:31.684479 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 565/1138] 10.222.0.33 () {60 vars in 1132 bytes} [Thu Sep 1 14:47:31 2022] POST /identity/v3/users => generated 286 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 14:47:33.621902 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27df899-c1fb-4441-8cb8-6807d5cd9596 None None] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:47:33.622497 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27df899-c1fb-4441-8cb8-6807d5cd9596 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:47:33.622497 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27df899-c1fb-4441-8cb8-6807d5cd9596 None None] PATH_INFO: `/` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:47:33.623800 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 574/1139] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 1 14:47:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:47:33.628813 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3764610-be24-46c3-93e3-e7d67663eac4 None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:47:33.629246 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3764610-be24-46c3-93e3-e7d67663eac4 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:47:33.629351 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3764610-be24-46c3-93e3-e7d67663eac4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:47:33.647028 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-d3764610-be24-46c3-93e3-e7d67663eac4 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:47:33.648213 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-d3764610-be24-46c3-93e3-e7d67663eac4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:47:33.713270 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 566/1140] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 1 14:47:33 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 14:47:33.980556 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27df899-c1fb-4441-8cb8-6807d5cd9596 None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:47:34.059965 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:47:34.079471 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27df899-c1fb-4441-8cb8-6807d5cd9596 None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:47:34.112023 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50685a1-802c-4950-a28b-a028ffd5f572 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:47:34.112631 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50685a1-802c-4950-a28b-a028ffd5f572 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:47:34.112907 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50685a1-802c-4950-a28b-a028ffd5f572 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:47:34.113174 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50685a1-802c-4950-a28b-a028ffd5f572 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:47:34.122889 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50685a1-802c-4950-a28b-a028ffd5f572 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 14:47:34.128098 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application [None req-a50685a1-802c-4950-a28b-a028ffd5f572 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 14:47:34.128098 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:47:34.128098 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:47:34.128098 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:47:34.128098 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:47:34.128098 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:47:34.128098 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:47:34.128098 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:47:34.128098 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:47:34.128098 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:47:34.128098 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:47:34.128098 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:47:34.128098 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 14:47:34.128098 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 14:47:34.128098 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 14:47:34.128098 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 14:47:34.129317 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 14:47:34.129317 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 14:47:34.129317 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 14:47:34.129317 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 14:47:34.129317 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:47:34.129317 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:47:34.129317 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:47:34.129317 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:47:34.129317 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:47:34.129317 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:47:34.129317 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:47:34.129317 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application with Lock( Sep 01 14:47:34.129317 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:47:34.129317 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:47:34.129317 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:47:34.129317 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:47:34.130502 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:47:34.130502 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:47:34.130502 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:47:34.130502 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:47:34.130502 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 14:47:34.130502 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 14:47:34.130502 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 14:47:34.130502 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 14:47:34.130502 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 14:47:34.130502 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 14:47:34.130502 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 14:47:34.130502 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Sep 01 14:47:34.130502 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 575/1141] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Sep 1 14:47:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:47:34.135147 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3764610-be24-46c3-93e3-e7d67663eac4 None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:47:34.221713 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:47:34.230925 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3764610-be24-46c3-93e3-e7d67663eac4 None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:47:34.263970 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67601f1-1e05-44ef-80b5-7024b8e071ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:47:34.264558 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67601f1-1e05-44ef-80b5-7024b8e071ca None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:47:34.264834 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67601f1-1e05-44ef-80b5-7024b8e071ca None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:47:34.265105 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67601f1-1e05-44ef-80b5-7024b8e071ca None admin] PATH_INFO: `/v3/roles` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:47:34.265762 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67601f1-1e05-44ef-80b5-7024b8e071ca None admin] RBAC: Adding query filter params (name=admin) {{(pid=81341) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:47:34.266061 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67601f1-1e05-44ef-80b5-7024b8e071ca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:47:34.267462 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67601f1-1e05-44ef-80b5-7024b8e071ca None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:47:34.275731 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 567/1142] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Sep 1 14:47:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:47:34.281262 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50685a1-802c-4950-a28b-a028ffd5f572 None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:47:34.376863 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50685a1-802c-4950-a28b-a028ffd5f572 None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:47:34.409677 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a4e30c-ef86-4c63-aaaf-1b07dd4b6d61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:47:34.410296 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a4e30c-ef86-4c63-aaaf-1b07dd4b6d61 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:47:34.410574 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a4e30c-ef86-4c63-aaaf-1b07dd4b6d61 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:47:34.410842 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a4e30c-ef86-4c63-aaaf-1b07dd4b6d61 None admin] PATH_INFO: `/v3/users/1750f6c051484ab49891320493040eca` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:47:34.421397 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:47:34.423897 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a4e30c-ef86-4c63-aaaf-1b07dd4b6d61 None admin] RBAC: Authorizing `identity:get_user(user_id=1750f6c051484ab49891320493040eca)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:47:34.425645 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a4e30c-ef86-4c63-aaaf-1b07dd4b6d61 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:47:34.427490 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 576/1143] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Sep 1 14:47:34 2022] GET /identity/v3/users/1750f6c051484ab49891320493040eca => generated 286 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:47:34.433128 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67601f1-1e05-44ef-80b5-7024b8e071ca None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:47:34.528791 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67601f1-1e05-44ef-80b5-7024b8e071ca None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:47:34.560509 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cd5930-50f4-41db-bba4-45e0d8e65510 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:47:34.561090 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cd5930-50f4-41db-bba4-45e0d8e65510 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:47:34.561364 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cd5930-50f4-41db-bba4-45e0d8e65510 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:47:34.561633 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cd5930-50f4-41db-bba4-45e0d8e65510 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:47:34.584642 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:47:34.626124 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0cd5930-50f4-41db-bba4-45e0d8e65510 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:47:34.629070 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0cd5930-50f4-41db-bba4-45e0d8e65510 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:47:34.721999 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 568/1144] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Sep 1 14:47:34 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 14:47:34.727806 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a4e30c-ef86-4c63-aaaf-1b07dd4b6d61 None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:47:34.824250 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a4e30c-ef86-4c63-aaaf-1b07dd4b6d61 None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:47:34.857751 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8be693-1bc4-42ec-8fde-124d69b9c18b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:47:34.858398 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8be693-1bc4-42ec-8fde-124d69b9c18b None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:47:34.858679 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8be693-1bc4-42ec-8fde-124d69b9c18b None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:47:34.858946 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8be693-1bc4-42ec-8fde-124d69b9c18b None admin] PATH_INFO: `/v3/projects/d712039e544943c6acbb2fa318259f64` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:47:34.860077 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8be693-1bc4-42ec-8fde-124d69b9c18b None admin] RBAC: Authorizing `identity:get_project(project_id=d712039e544943c6acbb2fa318259f64)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:47:34.861497 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8be693-1bc4-42ec-8fde-124d69b9c18b None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:47:34.863676 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 577/1145] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Sep 1 14:47:34 2022] GET /identity/v3/projects/d712039e544943c6acbb2fa318259f64 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:47:34.869541 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cd5930-50f4-41db-bba4-45e0d8e65510 None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:47:34.921829 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:47:34.965379 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cd5930-50f4-41db-bba4-45e0d8e65510 None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:47:34.998312 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66e89dc-baa4-4460-9d00-9aac051ff525 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:47:34.998894 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66e89dc-baa4-4460-9d00-9aac051ff525 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:47:34.999173 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66e89dc-baa4-4460-9d00-9aac051ff525 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:47:34.999443 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66e89dc-baa4-4460-9d00-9aac051ff525 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:47:35.000153 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66e89dc-baa4-4460-9d00-9aac051ff525 None admin] RBAC: Adding query filter params (role.id=2a2adb769e55476b82e5014897e0a9c0, scope.project.id=d712039e544943c6acbb2fa318259f64, user.id=1750f6c051484ab49891320493040eca) {{(pid=81341) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:47:35.000457 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66e89dc-baa4-4460-9d00-9aac051ff525 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:47:35.001887 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66e89dc-baa4-4460-9d00-9aac051ff525 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:47:35.007138 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 569/1146] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Sep 1 14:47:34 2022] GET /identity/v3/role_assignments?user.id=1750f6c051484ab49891320493040eca&scope.project.id=d712039e544943c6acbb2fa318259f64&role.id=2a2adb769e55476b82e5014897e0a9c0 => generated 259 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:47:36.861513 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63bf818-5750-4b6a-b807-cdc0b7d05a54 None None] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:47:36.862560 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63bf818-5750-4b6a-b807-cdc0b7d05a54 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:47:36.862560 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63bf818-5750-4b6a-b807-cdc0b7d05a54 None None] PATH_INFO: `/` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:47:36.863908 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 578/1147] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 1 14:47:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:47:36.869038 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ab4a8b-62d0-4e1b-a75d-da98b502298f None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:47:36.869449 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ab4a8b-62d0-4e1b-a75d-da98b502298f None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:47:36.869573 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ab4a8b-62d0-4e1b-a75d-da98b502298f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:47:36.887762 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-96ab4a8b-62d0-4e1b-a75d-da98b502298f None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:47:36.888937 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-96ab4a8b-62d0-4e1b-a75d-da98b502298f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:47:36.952404 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 570/1148] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 1 14:47:36 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 14:47:37.213041 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63bf818-5750-4b6a-b807-cdc0b7d05a54 None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:47:37.277108 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:47:37.309774 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63bf818-5750-4b6a-b807-cdc0b7d05a54 None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:47:37.341520 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6638bd47-4613-4ed0-8225-75bef4283c15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:47:37.342118 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6638bd47-4613-4ed0-8225-75bef4283c15 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:47:37.342421 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6638bd47-4613-4ed0-8225-75bef4283c15 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:47:37.342687 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6638bd47-4613-4ed0-8225-75bef4283c15 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:47:37.352777 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6638bd47-4613-4ed0-8225-75bef4283c15 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 14:47:37.357462 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application [None req-6638bd47-4613-4ed0-8225-75bef4283c15 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 14:47:37.357462 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:47:37.357462 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:47:37.357462 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:47:37.357462 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:47:37.357462 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:47:37.357462 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:47:37.357462 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:47:37.357462 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:47:37.357462 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:47:37.357462 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:47:37.357462 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:47:37.357462 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 14:47:37.357462 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 14:47:37.357462 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 14:47:37.357462 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 14:47:37.358599 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 14:47:37.358599 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 14:47:37.358599 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 14:47:37.358599 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 14:47:37.358599 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:47:37.358599 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:47:37.358599 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:47:37.358599 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:47:37.358599 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:47:37.358599 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:47:37.358599 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:47:37.358599 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application with Lock( Sep 01 14:47:37.358599 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:47:37.358599 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:47:37.358599 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:47:37.358599 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:47:37.359839 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:47:37.359839 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:47:37.359839 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:47:37.359839 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:47:37.359839 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 14:47:37.359839 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 14:47:37.359839 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 14:47:37.359839 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 14:47:37.359839 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 14:47:37.359839 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 14:47:37.359839 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 14:47:37.359839 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Sep 01 14:47:37.359839 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 579/1149] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Sep 1 14:47:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 146 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:47:37.364889 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ab4a8b-62d0-4e1b-a75d-da98b502298f None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:47:37.410789 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:47:37.459569 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ab4a8b-62d0-4e1b-a75d-da98b502298f None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:47:37.491829 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ebc5ca-eecf-4b7f-b982-3e9e40868039 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:47:37.492362 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ebc5ca-eecf-4b7f-b982-3e9e40868039 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:47:37.492646 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ebc5ca-eecf-4b7f-b982-3e9e40868039 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:47:37.492914 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ebc5ca-eecf-4b7f-b982-3e9e40868039 None admin] PATH_INFO: `/v3/roles` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:47:37.493567 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ebc5ca-eecf-4b7f-b982-3e9e40868039 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81341) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:47:37.493865 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ebc5ca-eecf-4b7f-b982-3e9e40868039 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:47:37.495266 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ebc5ca-eecf-4b7f-b982-3e9e40868039 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:47:37.502976 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 571/1150] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Sep 1 14:47:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:47:37.510195 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6638bd47-4613-4ed0-8225-75bef4283c15 None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:47:37.607386 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6638bd47-4613-4ed0-8225-75bef4283c15 None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:47:37.632129 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:47:37.639929 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca499546-4302-4e19-94bd-3f5da3114944 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:47:37.640543 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca499546-4302-4e19-94bd-3f5da3114944 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:47:37.640817 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca499546-4302-4e19-94bd-3f5da3114944 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:47:37.641084 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca499546-4302-4e19-94bd-3f5da3114944 None admin] PATH_INFO: `/v3/users/1750f6c051484ab49891320493040eca` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:47:37.642176 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca499546-4302-4e19-94bd-3f5da3114944 None admin] RBAC: Authorizing `identity:get_user(user_id=1750f6c051484ab49891320493040eca)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:47:37.644122 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca499546-4302-4e19-94bd-3f5da3114944 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:47:37.645666 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 580/1151] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Sep 1 14:47:37 2022] GET /identity/v3/users/1750f6c051484ab49891320493040eca => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:47:37.651570 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ebc5ca-eecf-4b7f-b982-3e9e40868039 None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:47:37.748105 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ebc5ca-eecf-4b7f-b982-3e9e40868039 None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:47:37.757759 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:47:37.780418 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1046e8a3-492b-4664-b275-320be453629f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:47:37.781029 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1046e8a3-492b-4664-b275-320be453629f None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:47:37.781302 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1046e8a3-492b-4664-b275-320be453629f None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:47:37.781566 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1046e8a3-492b-4664-b275-320be453629f None admin] PATH_INFO: `/v3/projects/d712039e544943c6acbb2fa318259f64` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:47:37.782693 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1046e8a3-492b-4664-b275-320be453629f None admin] RBAC: Authorizing `identity:get_project(project_id=d712039e544943c6acbb2fa318259f64)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:47:37.784120 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1046e8a3-492b-4664-b275-320be453629f None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:47:37.786321 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 572/1152] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Sep 1 14:47:37 2022] GET /identity/v3/projects/d712039e544943c6acbb2fa318259f64 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:47:37.792199 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca499546-4302-4e19-94bd-3f5da3114944 None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:47:37.886970 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca499546-4302-4e19-94bd-3f5da3114944 None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:47:37.918852 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae086bc-a44b-4ab4-8955-dff42cf8ef90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:47:37.919478 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae086bc-a44b-4ab4-8955-dff42cf8ef90 None admin] REQUEST_METHOD: `PUT` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:47:37.919752 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae086bc-a44b-4ab4-8955-dff42cf8ef90 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:47:37.920017 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae086bc-a44b-4ab4-8955-dff42cf8ef90 None admin] PATH_INFO: `/v3/projects/d712039e544943c6acbb2fa318259f64/users/1750f6c051484ab49891320493040eca/roles/2a2adb769e55476b82e5014897e0a9c0` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:47:37.921848 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae086bc-a44b-4ab4-8955-dff42cf8ef90 None admin] RBAC: Authorizing `identity:create_grant(project_id=d712039e544943c6acbb2fa318259f64, user_id=1750f6c051484ab49891320493040eca, role_id=2a2adb769e55476b82e5014897e0a9c0)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:47:37.923285 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae086bc-a44b-4ab4-8955-dff42cf8ef90 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:47:37.935741 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 581/1153] 10.222.0.33 () {58 vars in 1439 bytes} [Thu Sep 1 14:47:37 2022] PUT /identity/v3/projects/d712039e544943c6acbb2fa318259f64/users/1750f6c051484ab49891320493040eca/roles/2a2adb769e55476b82e5014897e0a9c0 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:47:39.798470 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7859ba22-90d1-4ba8-ab7e-1eff3c0d375a None None] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:47:39.798997 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7859ba22-90d1-4ba8-ab7e-1eff3c0d375a None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:47:39.798997 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7859ba22-90d1-4ba8-ab7e-1eff3c0d375a None None] PATH_INFO: `/` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:47:39.800790 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 573/1154] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 1 14:47:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:47:39.805819 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a17639-f26e-406a-8472-bac5b6a04f8b None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:47:39.806180 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a17639-f26e-406a-8472-bac5b6a04f8b None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:47:39.806399 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a17639-f26e-406a-8472-bac5b6a04f8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:47:39.825366 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-49a17639-f26e-406a-8472-bac5b6a04f8b None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:47:39.826591 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-49a17639-f26e-406a-8472-bac5b6a04f8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:47:39.875071 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:47:39.892276 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 582/1155] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 1 14:47:39 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 14:47:40.160720 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7859ba22-90d1-4ba8-ab7e-1eff3c0d375a None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:47:40.257217 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7859ba22-90d1-4ba8-ab7e-1eff3c0d375a None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:47:40.288879 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f730d7b8-8a6b-4995-8262-3f22ee77d445 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:47:40.289481 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f730d7b8-8a6b-4995-8262-3f22ee77d445 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:47:40.289756 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f730d7b8-8a6b-4995-8262-3f22ee77d445 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:47:40.290022 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f730d7b8-8a6b-4995-8262-3f22ee77d445 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:47:40.300989 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f730d7b8-8a6b-4995-8262-3f22ee77d445 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 14:47:40.305663 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application [None req-f730d7b8-8a6b-4995-8262-3f22ee77d445 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 14:47:40.305663 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:47:40.305663 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:47:40.305663 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:47:40.305663 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:47:40.305663 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:47:40.305663 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:47:40.305663 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:47:40.305663 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:47:40.305663 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:47:40.305663 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:47:40.305663 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:47:40.305663 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 14:47:40.305663 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 14:47:40.305663 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 14:47:40.305663 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 14:47:40.306640 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 14:47:40.306640 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 14:47:40.306640 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 14:47:40.306640 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 14:47:40.306640 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:47:40.306640 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:47:40.306640 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:47:40.306640 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:47:40.306640 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:47:40.306640 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:47:40.306640 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:47:40.306640 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application with Lock( Sep 01 14:47:40.306640 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:47:40.306640 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:47:40.306640 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:47:40.306640 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:47:40.307905 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:47:40.307905 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:47:40.307905 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:47:40.307905 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:47:40.307905 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 14:47:40.307905 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 14:47:40.307905 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 14:47:40.307905 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 14:47:40.307905 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 14:47:40.307905 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 14:47:40.307905 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 14:47:40.307905 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Sep 01 14:47:40.307905 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 574/1156] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Sep 1 14:47:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:47:40.312635 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a17639-f26e-406a-8472-bac5b6a04f8b None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:47:40.406136 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a17639-f26e-406a-8472-bac5b6a04f8b None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:47:40.439285 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92c8f4c-c04b-45e3-8565-baf573b71712 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:47:40.439818 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92c8f4c-c04b-45e3-8565-baf573b71712 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:47:40.440094 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92c8f4c-c04b-45e3-8565-baf573b71712 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:47:40.440360 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92c8f4c-c04b-45e3-8565-baf573b71712 None admin] PATH_INFO: `/v3/roles` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:47:40.441041 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92c8f4c-c04b-45e3-8565-baf573b71712 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:47:40.441342 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92c8f4c-c04b-45e3-8565-baf573b71712 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:47:40.442730 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92c8f4c-c04b-45e3-8565-baf573b71712 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:47:40.450692 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 583/1157] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Sep 1 14:47:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:47:40.456184 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f730d7b8-8a6b-4995-8262-3f22ee77d445 None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:47:40.494863 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:47:40.551979 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f730d7b8-8a6b-4995-8262-3f22ee77d445 None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:47:40.583907 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234cc3d7-3908-4545-ac46-a53ca0bc2df0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:47:40.584503 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234cc3d7-3908-4545-ac46-a53ca0bc2df0 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:47:40.584778 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234cc3d7-3908-4545-ac46-a53ca0bc2df0 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:47:40.585046 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234cc3d7-3908-4545-ac46-a53ca0bc2df0 None admin] PATH_INFO: `/v3/users/1750f6c051484ab49891320493040eca` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:47:40.586138 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234cc3d7-3908-4545-ac46-a53ca0bc2df0 None admin] RBAC: Authorizing `identity:get_user(user_id=1750f6c051484ab49891320493040eca)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:47:40.587602 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234cc3d7-3908-4545-ac46-a53ca0bc2df0 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:47:40.589115 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 575/1158] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Sep 1 14:47:40 2022] GET /identity/v3/users/1750f6c051484ab49891320493040eca => generated 286 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:47:40.594852 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92c8f4c-c04b-45e3-8565-baf573b71712 None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:47:40.642598 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:47:40.697439 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92c8f4c-c04b-45e3-8565-baf573b71712 None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:47:40.730133 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6cd169-2f04-4954-a1e7-dcac2a723c93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:47:40.730749 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6cd169-2f04-4954-a1e7-dcac2a723c93 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:47:40.731026 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6cd169-2f04-4954-a1e7-dcac2a723c93 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:47:40.731291 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6cd169-2f04-4954-a1e7-dcac2a723c93 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:47:40.795738 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6cd169-2f04-4954-a1e7-dcac2a723c93 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:47:40.797177 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6cd169-2f04-4954-a1e7-dcac2a723c93 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:47:40.843113 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:47:40.898755 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 584/1159] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Sep 1 14:47:40 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 14:47:40.904777 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234cc3d7-3908-4545-ac46-a53ca0bc2df0 None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:47:40.999857 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234cc3d7-3908-4545-ac46-a53ca0bc2df0 None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:47:41.006834 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:47:41.033033 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e0322a-b3e6-4258-9eae-9e4c748377d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:47:41.033636 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e0322a-b3e6-4258-9eae-9e4c748377d9 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:47:41.033912 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e0322a-b3e6-4258-9eae-9e4c748377d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:47:41.034182 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e0322a-b3e6-4258-9eae-9e4c748377d9 None admin] PATH_INFO: `/v3/projects/d712039e544943c6acbb2fa318259f64` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:47:41.035361 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e0322a-b3e6-4258-9eae-9e4c748377d9 None admin] RBAC: Authorizing `identity:get_project(project_id=d712039e544943c6acbb2fa318259f64)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:47:41.036805 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e0322a-b3e6-4258-9eae-9e4c748377d9 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:47:41.038405 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 576/1160] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Sep 1 14:47:40 2022] GET /identity/v3/projects/d712039e544943c6acbb2fa318259f64 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:47:41.044447 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6cd169-2f04-4954-a1e7-dcac2a723c93 None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:47:41.140230 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6cd169-2f04-4954-a1e7-dcac2a723c93 None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:47:41.172962 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341bc85d-408b-4c3a-91e2-0191cf2fc15c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:47:41.173546 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341bc85d-408b-4c3a-91e2-0191cf2fc15c None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:47:41.173825 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341bc85d-408b-4c3a-91e2-0191cf2fc15c None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:47:41.174092 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341bc85d-408b-4c3a-91e2-0191cf2fc15c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:47:41.174842 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341bc85d-408b-4c3a-91e2-0191cf2fc15c None admin] RBAC: Adding query filter params (role.id=2a2adb769e55476b82e5014897e0a9c0, scope.project.id=d712039e544943c6acbb2fa318259f64, user.id=1750f6c051484ab49891320493040eca) {{(pid=81342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:47:41.175149 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341bc85d-408b-4c3a-91e2-0191cf2fc15c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:47:41.177080 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341bc85d-408b-4c3a-91e2-0191cf2fc15c None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:47:41.182635 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 585/1161] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Sep 1 14:47:41 2022] GET /identity/v3/role_assignments?user.id=1750f6c051484ab49891320493040eca&scope.project.id=d712039e544943c6acbb2fa318259f64&role.id=2a2adb769e55476b82e5014897e0a9c0 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:47:43.060478 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abda4a2a-d183-4867-b42c-795d3e4ed1b8 None None] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:47:43.061017 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abda4a2a-d183-4867-b42c-795d3e4ed1b8 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:47:43.061017 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abda4a2a-d183-4867-b42c-795d3e4ed1b8 None None] PATH_INFO: `/` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:47:43.062378 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 577/1162] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 1 14:47:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:47:43.067405 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d23ef50-5294-459c-9737-0006b5814cb8 None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:47:43.067786 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d23ef50-5294-459c-9737-0006b5814cb8 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:47:43.067937 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d23ef50-5294-459c-9737-0006b5814cb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:47:43.085693 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-8d23ef50-5294-459c-9737-0006b5814cb8 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:47:43.086896 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-8d23ef50-5294-459c-9737-0006b5814cb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:47:43.093960 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:47:43.150447 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 586/1163] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 1 14:47:43 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 14:47:43.417060 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abda4a2a-d183-4867-b42c-795d3e4ed1b8 None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:47:43.515146 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abda4a2a-d183-4867-b42c-795d3e4ed1b8 None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:47:43.547760 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15348d5b-d481-4014-8480-3918347dcf41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:47:43.548349 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15348d5b-d481-4014-8480-3918347dcf41 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:47:43.548630 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15348d5b-d481-4014-8480-3918347dcf41 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:47:43.548909 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15348d5b-d481-4014-8480-3918347dcf41 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:47:43.590419 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:47:43.615640 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15348d5b-d481-4014-8480-3918347dcf41 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:47:43.617107 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15348d5b-d481-4014-8480-3918347dcf41 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:47:43.712830 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 578/1164] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Sep 1 14:47:43 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 14:47:43.718690 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d23ef50-5294-459c-9737-0006b5814cb8 None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:47:43.813411 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d23ef50-5294-459c-9737-0006b5814cb8 None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:47:43.846573 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e81f8e-f44d-424b-a6c6-e66c46a84c38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:47:43.847180 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e81f8e-f44d-424b-a6c6-e66c46a84c38 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:47:43.847456 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e81f8e-f44d-424b-a6c6-e66c46a84c38 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:47:43.847722 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e81f8e-f44d-424b-a6c6-e66c46a84c38 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:47:43.854124 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e81f8e-f44d-424b-a6c6-e66c46a84c38 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:47:43.855574 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e81f8e-f44d-424b-a6c6-e66c46a84c38 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:47:43.866596 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application [None req-f8e81f8e-f44d-424b-a6c6-e66c46a84c38 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 01 14:47:43.866596 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:47:43.866596 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 01 14:47:43.866596 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 01 14:47:43.866596 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 14:47:43.866596 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 14:47:43.866596 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 14:47:43.866596 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 14:47:43.866596 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Sep 01 14:47:43.866596 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Sep 01 14:47:43.866596 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 14:47:43.866596 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Sep 01 14:47:43.866596 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:47:43.866596 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:47:43.866596 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:47:43.866596 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:47:43.866596 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:47:43.867786 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:47:43.867786 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:47:43.867786 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:47:43.867786 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:47:43.867786 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:47:43.867786 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:47:43.867786 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 14:47:43.867786 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 14:47:43.867786 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 14:47:43.867786 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 14:47:43.867786 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:47:43.867786 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:47:43.867786 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:47:43.867786 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:47:43.867786 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:47:43.867786 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:47:43.868680 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:47:43.868680 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application with Lock( Sep 01 14:47:43.868680 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:47:43.868680 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:47:43.868680 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:47:43.868680 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:47:43.868680 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:47:43.868680 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:47:43.868680 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:47:43.868680 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:47:43.868680 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 01 14:47:43.868680 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 14:47:43.868680 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 01 14:47:43.868680 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Sep 01 14:47:43.868680 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 587/1165] 10.222.0.33 () {56 vars in 1117 bytes} [Thu Sep 1 14:47:43 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:47:43.873372 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15348d5b-d481-4014-8480-3918347dcf41 None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:47:43.944819 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:47:43.969267 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15348d5b-d481-4014-8480-3918347dcf41 None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:47:44.001475 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd41aee-7519-4d9a-9c75-b79730112707 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:47:44.002065 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd41aee-7519-4d9a-9c75-b79730112707 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:47:44.002371 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd41aee-7519-4d9a-9c75-b79730112707 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:47:44.002641 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd41aee-7519-4d9a-9c75-b79730112707 None admin] PATH_INFO: `/v3/domains` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:47:44.003274 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd41aee-7519-4d9a-9c75-b79730112707 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81341) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:47:44.004069 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd41aee-7519-4d9a-9c75-b79730112707 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:47:44.005433 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd41aee-7519-4d9a-9c75-b79730112707 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:47:44.012270 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 579/1166] 10.222.0.33 () {56 vars in 1115 bytes} [Thu Sep 1 14:47:43 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:47:45.882224 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4449095b-7a55-4eba-92b1-faa5412c5a8b None None] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:47:45.882767 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4449095b-7a55-4eba-92b1-faa5412c5a8b None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:47:45.882767 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4449095b-7a55-4eba-92b1-faa5412c5a8b None None] PATH_INFO: `/` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:47:45.884102 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 588/1167] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 1 14:47:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:47:45.889098 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2be5180-3299-4bfb-896b-76dc382bbd98 None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:47:45.889471 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2be5180-3299-4bfb-896b-76dc382bbd98 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:47:45.889662 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2be5180-3299-4bfb-896b-76dc382bbd98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:47:45.907457 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-e2be5180-3299-4bfb-896b-76dc382bbd98 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:47:45.908643 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-e2be5180-3299-4bfb-896b-76dc382bbd98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:47:45.972439 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 580/1168] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 1 14:47:45 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 14:47:46.236533 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4449095b-7a55-4eba-92b1-faa5412c5a8b None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:47:46.239943 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:47:46.336295 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4449095b-7a55-4eba-92b1-faa5412c5a8b None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:47:46.369392 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7d6e1d-8db2-4c27-ae48-5dd86cfc8e46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:47:46.369970 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7d6e1d-8db2-4c27-ae48-5dd86cfc8e46 None admin] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:47:46.370269 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7d6e1d-8db2-4c27-ae48-5dd86cfc8e46 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:47:46.370543 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7d6e1d-8db2-4c27-ae48-5dd86cfc8e46 None admin] PATH_INFO: `/v3/domains` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:47:46.371205 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7d6e1d-8db2-4c27-ae48-5dd86cfc8e46 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:47:46.372565 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7d6e1d-8db2-4c27-ae48-5dd86cfc8e46 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:47:46.387523 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 589/1169] 10.222.0.33 () {60 vars in 1138 bytes} [Thu Sep 1 14:47:46 2022] POST /identity/v3/domains => generated 267 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:47:48.292468 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdaacdc-bd7d-4a72-87fc-a87140c16fee None None] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:47:48.292996 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdaacdc-bd7d-4a72-87fc-a87140c16fee None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:47:48.292996 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdaacdc-bd7d-4a72-87fc-a87140c16fee None None] PATH_INFO: `/` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:47:48.294382 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 581/1170] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 1 14:47:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:47:48.299416 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b90748-40e1-4c57-b280-dd8db5624266 None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:47:48.299834 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b90748-40e1-4c57-b280-dd8db5624266 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:47:48.299951 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b90748-40e1-4c57-b280-dd8db5624266 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:47:48.317962 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-42b90748-40e1-4c57-b280-dd8db5624266 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:47:48.319191 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-42b90748-40e1-4c57-b280-dd8db5624266 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:47:48.373209 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:47:48.385210 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 590/1171] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 1 14:47:48 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 14:47:48.653872 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdaacdc-bd7d-4a72-87fc-a87140c16fee None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:47:48.718846 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:47:48.752282 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdaacdc-bd7d-4a72-87fc-a87140c16fee None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:47:48.784713 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799da410-255b-4a2d-8ad1-8357b27d8af2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:47:48.785327 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799da410-255b-4a2d-8ad1-8357b27d8af2 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:47:48.785606 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799da410-255b-4a2d-8ad1-8357b27d8af2 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:47:48.785877 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799da410-255b-4a2d-8ad1-8357b27d8af2 None admin] PATH_INFO: `/v3/domains/167728a9a734425f8905b3f213c7fbdf` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:47:48.795857 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799da410-255b-4a2d-8ad1-8357b27d8af2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=167728a9a734425f8905b3f213c7fbdf)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:47:48.797302 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799da410-255b-4a2d-8ad1-8357b27d8af2 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:47:48.798885 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 582/1172] 10.222.0.33 () {56 vars in 1183 bytes} [Thu Sep 1 14:47:48 2022] GET /identity/v3/domains/167728a9a734425f8905b3f213c7fbdf => generated 267 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:47:48.805239 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b90748-40e1-4c57-b280-dd8db5624266 None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:47:48.899850 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b90748-40e1-4c57-b280-dd8db5624266 None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:47:48.932270 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bd398f-1b0f-46b5-b6e2-5fab45ae543f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:47:48.932802 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bd398f-1b0f-46b5-b6e2-5fab45ae543f None admin] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:47:48.933080 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bd398f-1b0f-46b5-b6e2-5fab45ae543f None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:47:48.933347 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bd398f-1b0f-46b5-b6e2-5fab45ae543f None admin] PATH_INFO: `/v3/projects` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:47:48.933986 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bd398f-1b0f-46b5-b6e2-5fab45ae543f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:47:48.935434 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bd398f-1b0f-46b5-b6e2-5fab45ae543f None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:47:48.961053 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 591/1173] 10.222.0.33 () {60 vars in 1141 bytes} [Thu Sep 1 14:47:48 2022] POST /identity/v3/projects => generated 361 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 14:47:50.867886 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be424f5-3592-46a9-a749-6f93429c8b2c None None] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:47:50.868422 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be424f5-3592-46a9-a749-6f93429c8b2c None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:47:50.868422 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be424f5-3592-46a9-a749-6f93429c8b2c None None] PATH_INFO: `/` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:47:50.869741 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 583/1174] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 1 14:47:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 14:47:50.874838 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70144230-6865-4caf-9650-a5bef2fac1b9 None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:47:50.875262 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70144230-6865-4caf-9650-a5bef2fac1b9 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:47:50.875372 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70144230-6865-4caf-9650-a5bef2fac1b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:47:50.892906 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-70144230-6865-4caf-9650-a5bef2fac1b9 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:47:50.894073 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-70144230-6865-4caf-9650-a5bef2fac1b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:47:50.933499 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:47:50.960131 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 592/1175] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 1 14:47:50 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 14:47:51.229669 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be424f5-3592-46a9-a749-6f93429c8b2c None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:47:51.327073 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be424f5-3592-46a9-a749-6f93429c8b2c None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:47:51.351992 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:47:51.359662 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c99112b-0d43-4706-8e83-82708bd6dc2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:47:51.360277 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c99112b-0d43-4706-8e83-82708bd6dc2c None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:47:51.360554 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c99112b-0d43-4706-8e83-82708bd6dc2c None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:47:51.360824 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c99112b-0d43-4706-8e83-82708bd6dc2c None admin] PATH_INFO: `/v3/domains/167728a9a734425f8905b3f213c7fbdf` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:47:51.361923 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c99112b-0d43-4706-8e83-82708bd6dc2c None admin] RBAC: Authorizing `identity:get_domain(domain_id=167728a9a734425f8905b3f213c7fbdf)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:47:51.363449 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c99112b-0d43-4706-8e83-82708bd6dc2c None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:47:51.364979 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 584/1176] 10.222.0.33 () {56 vars in 1183 bytes} [Thu Sep 1 14:47:51 2022] GET /identity/v3/domains/167728a9a734425f8905b3f213c7fbdf => generated 267 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:47:51.370759 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70144230-6865-4caf-9650-a5bef2fac1b9 None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:47:51.465802 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70144230-6865-4caf-9650-a5bef2fac1b9 None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:47:51.497751 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1f85d1-89dd-45be-80a8-93515d155868 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:47:51.498306 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1f85d1-89dd-45be-80a8-93515d155868 None admin] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:47:51.498586 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1f85d1-89dd-45be-80a8-93515d155868 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:47:51.498857 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1f85d1-89dd-45be-80a8-93515d155868 None admin] PATH_INFO: `/v3/users` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:47:51.499503 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1f85d1-89dd-45be-80a8-93515d155868 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:47:51.500919 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1f85d1-89dd-45be-80a8-93515d155868 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:47:51.521734 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 593/1177] 10.222.0.33 () {60 vars in 1132 bytes} [Thu Sep 1 14:47:51 2022] POST /identity/v3/users => generated 311 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:47:53.427507 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123a3719-1193-4c8a-b001-9478229ab58b None None] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:47:53.429311 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123a3719-1193-4c8a-b001-9478229ab58b None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:47:53.429311 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123a3719-1193-4c8a-b001-9478229ab58b None None] PATH_INFO: `/` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:47:53.429550 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 585/1178] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 1 14:47:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:47:53.434610 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded48817-fd17-4c0c-b63a-5ea661b1aadb None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:47:53.435016 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded48817-fd17-4c0c-b63a-5ea661b1aadb None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:47:53.435160 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded48817-fd17-4c0c-b63a-5ea661b1aadb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:47:53.453270 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-ded48817-fd17-4c0c-b63a-5ea661b1aadb None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:47:53.454490 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-ded48817-fd17-4c0c-b63a-5ea661b1aadb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:47:53.478131 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:47:53.520357 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 594/1179] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 1 14:47:53 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 14:47:53.788487 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123a3719-1193-4c8a-b001-9478229ab58b None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:47:53.886681 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123a3719-1193-4c8a-b001-9478229ab58b None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:47:53.918645 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62d87f0-049b-4bff-8dd4-df2efc574131 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:47:53.919253 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62d87f0-049b-4bff-8dd4-df2efc574131 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:47:53.919529 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62d87f0-049b-4bff-8dd4-df2efc574131 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:47:53.919797 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62d87f0-049b-4bff-8dd4-df2efc574131 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:47:53.931116 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62d87f0-049b-4bff-8dd4-df2efc574131 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 14:47:53.935825 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application [None req-c62d87f0-049b-4bff-8dd4-df2efc574131 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 14:47:53.935825 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:47:53.935825 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:47:53.935825 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:47:53.935825 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:47:53.935825 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:47:53.935825 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:47:53.935825 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:47:53.935825 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:47:53.935825 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:47:53.935825 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:47:53.935825 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:47:53.935825 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 14:47:53.935825 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 14:47:53.935825 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 14:47:53.935825 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 14:47:53.937043 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 14:47:53.937043 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 14:47:53.937043 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 14:47:53.937043 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 14:47:53.937043 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:47:53.937043 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:47:53.937043 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:47:53.937043 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:47:53.937043 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:47:53.937043 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:47:53.937043 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:47:53.937043 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application with Lock( Sep 01 14:47:53.937043 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:47:53.937043 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:47:53.937043 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:47:53.937043 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:47:53.938257 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:47:53.938257 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:47:53.938257 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:47:53.938257 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:47:53.938257 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 14:47:53.938257 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 14:47:53.938257 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 14:47:53.938257 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 14:47:53.938257 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 14:47:53.938257 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 14:47:53.938257 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 14:47:53.938257 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Sep 01 14:47:53.938257 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 586/1180] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Sep 1 14:47:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:47:53.943278 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded48817-fd17-4c0c-b63a-5ea661b1aadb None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:47:54.038850 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded48817-fd17-4c0c-b63a-5ea661b1aadb None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:47:54.071878 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adff204b-a1c7-4066-b411-f50847cf228a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:47:54.072464 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adff204b-a1c7-4066-b411-f50847cf228a None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:47:54.072737 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adff204b-a1c7-4066-b411-f50847cf228a None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:47:54.073001 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adff204b-a1c7-4066-b411-f50847cf228a None admin] PATH_INFO: `/v3/roles` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:47:54.073649 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adff204b-a1c7-4066-b411-f50847cf228a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:47:54.073942 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adff204b-a1c7-4066-b411-f50847cf228a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:47:54.075308 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adff204b-a1c7-4066-b411-f50847cf228a None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:47:54.082978 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 595/1181] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Sep 1 14:47:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:47:54.088675 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62d87f0-049b-4bff-8dd4-df2efc574131 None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:47:54.142918 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:47:54.185568 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62d87f0-049b-4bff-8dd4-df2efc574131 None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:47:54.217613 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6403c1a-c99d-4fe5-8c7a-aebe27eaf045 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:47:54.218218 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6403c1a-c99d-4fe5-8c7a-aebe27eaf045 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:47:54.218522 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6403c1a-c99d-4fe5-8c7a-aebe27eaf045 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:47:54.218789 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6403c1a-c99d-4fe5-8c7a-aebe27eaf045 None admin] PATH_INFO: `/v3/users/c19f22bfb4e54b44b6775f3c6ee80fc6` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:47:54.231125 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6403c1a-c99d-4fe5-8c7a-aebe27eaf045 None admin] RBAC: Authorizing `identity:get_user(user_id=c19f22bfb4e54b44b6775f3c6ee80fc6)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:47:54.232559 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6403c1a-c99d-4fe5-8c7a-aebe27eaf045 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:47:54.234133 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 587/1182] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Sep 1 14:47:54 2022] GET /identity/v3/users/c19f22bfb4e54b44b6775f3c6ee80fc6 => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:47:54.240502 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adff204b-a1c7-4066-b411-f50847cf228a None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:47:54.264643 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:47:54.337363 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adff204b-a1c7-4066-b411-f50847cf228a None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:47:54.369823 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd83716-cc8f-4551-ae4a-28ed2f96e617 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:47:54.370441 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd83716-cc8f-4551-ae4a-28ed2f96e617 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:47:54.370717 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd83716-cc8f-4551-ae4a-28ed2f96e617 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:47:54.370982 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd83716-cc8f-4551-ae4a-28ed2f96e617 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:47:54.434977 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd83716-cc8f-4551-ae4a-28ed2f96e617 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:47:54.436401 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd83716-cc8f-4551-ae4a-28ed2f96e617 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:47:54.458256 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:47:54.530993 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 596/1183] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Sep 1 14:47:54 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 14:47:54.537473 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6403c1a-c99d-4fe5-8c7a-aebe27eaf045 None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:47:54.632534 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6403c1a-c99d-4fe5-8c7a-aebe27eaf045 None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:47:54.646250 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:47:54.666737 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5a917a-cda6-4a8b-ba6c-8184a4260efa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:47:54.667355 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5a917a-cda6-4a8b-ba6c-8184a4260efa None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:47:54.667629 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5a917a-cda6-4a8b-ba6c-8184a4260efa None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:47:54.667894 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5a917a-cda6-4a8b-ba6c-8184a4260efa None admin] PATH_INFO: `/v3/projects/7a2da326ed3543b197ba594c52ad0f0c` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:47:54.668983 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5a917a-cda6-4a8b-ba6c-8184a4260efa None admin] RBAC: Authorizing `identity:get_project(project_id=7a2da326ed3543b197ba594c52ad0f0c)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:47:54.670439 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5a917a-cda6-4a8b-ba6c-8184a4260efa None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:47:54.672030 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 588/1184] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Sep 1 14:47:54 2022] GET /identity/v3/projects/7a2da326ed3543b197ba594c52ad0f0c => generated 361 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:47:54.678047 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd83716-cc8f-4551-ae4a-28ed2f96e617 None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:47:54.772955 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd83716-cc8f-4551-ae4a-28ed2f96e617 None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:47:54.796369 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:47:54.807716 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcc6d07-731a-4f9c-aecc-ecfe6277fd24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:47:54.808303 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcc6d07-731a-4f9c-aecc-ecfe6277fd24 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:47:54.808577 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcc6d07-731a-4f9c-aecc-ecfe6277fd24 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:47:54.808841 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcc6d07-731a-4f9c-aecc-ecfe6277fd24 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:47:54.809555 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbcc6d07-731a-4f9c-aecc-ecfe6277fd24 None admin] RBAC: Adding query filter params (role.id=2a2adb769e55476b82e5014897e0a9c0, scope.project.id=7a2da326ed3543b197ba594c52ad0f0c, user.id=c19f22bfb4e54b44b6775f3c6ee80fc6) {{(pid=81342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:47:54.809858 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbcc6d07-731a-4f9c-aecc-ecfe6277fd24 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:47:54.811298 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbcc6d07-731a-4f9c-aecc-ecfe6277fd24 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:47:54.816441 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 597/1185] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Sep 1 14:47:54 2022] GET /identity/v3/role_assignments?user.id=c19f22bfb4e54b44b6775f3c6ee80fc6&scope.project.id=7a2da326ed3543b197ba594c52ad0f0c&role.id=2a2adb769e55476b82e5014897e0a9c0 => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:47:56.671397 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05edb7e9-a29d-448b-80ea-99baeda9de70 None None] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:47:56.671815 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05edb7e9-a29d-448b-80ea-99baeda9de70 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:47:56.671919 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05edb7e9-a29d-448b-80ea-99baeda9de70 None None] PATH_INFO: `/` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:47:56.673245 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 589/1186] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 1 14:47:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:47:56.678316 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9f5865-17d7-48b2-89b5-4b259d46d272 None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:47:56.678735 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9f5865-17d7-48b2-89b5-4b259d46d272 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:47:56.678886 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9f5865-17d7-48b2-89b5-4b259d46d272 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:47:56.697370 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-1a9f5865-17d7-48b2-89b5-4b259d46d272 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:47:56.698606 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-1a9f5865-17d7-48b2-89b5-4b259d46d272 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:47:56.761837 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 598/1187] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 1 14:47:56 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 14:47:57.028314 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05edb7e9-a29d-448b-80ea-99baeda9de70 None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:47:57.125419 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05edb7e9-a29d-448b-80ea-99baeda9de70 None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:47:57.158309 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2feaba48-4464-4a9b-b3d4-d2df12819c24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:47:57.158919 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2feaba48-4464-4a9b-b3d4-d2df12819c24 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:47:57.159198 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2feaba48-4464-4a9b-b3d4-d2df12819c24 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:47:57.159468 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2feaba48-4464-4a9b-b3d4-d2df12819c24 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:47:57.169102 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2feaba48-4464-4a9b-b3d4-d2df12819c24 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 14:47:57.173832 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application [None req-2feaba48-4464-4a9b-b3d4-d2df12819c24 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 14:47:57.173832 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:47:57.173832 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:47:57.173832 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:47:57.173832 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:47:57.173832 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:47:57.173832 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:47:57.173832 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:47:57.173832 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:47:57.173832 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:47:57.173832 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:47:57.173832 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:47:57.173832 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 14:47:57.173832 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 14:47:57.173832 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 14:47:57.173832 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 14:47:57.174984 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 14:47:57.174984 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 14:47:57.174984 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 14:47:57.174984 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 14:47:57.174984 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:47:57.174984 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:47:57.174984 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:47:57.174984 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:47:57.174984 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:47:57.174984 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:47:57.174984 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:47:57.174984 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application with Lock( Sep 01 14:47:57.174984 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:47:57.174984 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:47:57.174984 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:47:57.174984 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:47:57.176447 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:47:57.176447 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:47:57.176447 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:47:57.176447 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:47:57.176447 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 14:47:57.176447 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 14:47:57.176447 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 14:47:57.176447 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 14:47:57.176447 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 14:47:57.176447 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 14:47:57.176447 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 14:47:57.176447 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Sep 01 14:47:57.176447 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 590/1188] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Sep 1 14:47:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:47:57.181355 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9f5865-17d7-48b2-89b5-4b259d46d272 None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:47:57.277143 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9f5865-17d7-48b2-89b5-4b259d46d272 None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:47:57.293400 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:47:57.311143 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d4cafe-bff7-476d-8be0-5cd0fd30bda4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:47:57.311739 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d4cafe-bff7-476d-8be0-5cd0fd30bda4 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:47:57.312014 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d4cafe-bff7-476d-8be0-5cd0fd30bda4 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:47:57.312282 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d4cafe-bff7-476d-8be0-5cd0fd30bda4 None admin] PATH_INFO: `/v3/roles` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:47:57.312940 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d4cafe-bff7-476d-8be0-5cd0fd30bda4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:47:57.313236 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d4cafe-bff7-476d-8be0-5cd0fd30bda4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:47:57.314623 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d4cafe-bff7-476d-8be0-5cd0fd30bda4 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:47:57.322779 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 599/1189] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Sep 1 14:47:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:47:57.329031 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2feaba48-4464-4a9b-b3d4-d2df12819c24 None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:47:57.370814 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:47:57.426656 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2feaba48-4464-4a9b-b3d4-d2df12819c24 None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:47:57.459382 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e763aa-bf18-43d9-94ee-6fbf857c6a7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:47:57.459989 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e763aa-bf18-43d9-94ee-6fbf857c6a7a None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:47:57.460270 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e763aa-bf18-43d9-94ee-6fbf857c6a7a None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:47:57.460543 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e763aa-bf18-43d9-94ee-6fbf857c6a7a None admin] PATH_INFO: `/v3/users/c19f22bfb4e54b44b6775f3c6ee80fc6` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:47:57.461661 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e763aa-bf18-43d9-94ee-6fbf857c6a7a None admin] RBAC: Authorizing `identity:get_user(user_id=c19f22bfb4e54b44b6775f3c6ee80fc6)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:47:57.463131 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e763aa-bf18-43d9-94ee-6fbf857c6a7a None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:47:57.464695 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 591/1190] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Sep 1 14:47:57 2022] GET /identity/v3/users/c19f22bfb4e54b44b6775f3c6ee80fc6 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:47:57.470360 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d4cafe-bff7-476d-8be0-5cd0fd30bda4 None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:47:57.565958 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d4cafe-bff7-476d-8be0-5cd0fd30bda4 None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:47:57.598906 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8773a584-fe93-478d-8d03-460ae5f41daa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:47:57.599516 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8773a584-fe93-478d-8d03-460ae5f41daa None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:47:57.599793 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8773a584-fe93-478d-8d03-460ae5f41daa None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:47:57.600061 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8773a584-fe93-478d-8d03-460ae5f41daa None admin] PATH_INFO: `/v3/projects/7a2da326ed3543b197ba594c52ad0f0c` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:47:57.601162 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8773a584-fe93-478d-8d03-460ae5f41daa None admin] RBAC: Authorizing `identity:get_project(project_id=7a2da326ed3543b197ba594c52ad0f0c)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:47:57.602617 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8773a584-fe93-478d-8d03-460ae5f41daa None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:47:57.604229 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 600/1191] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Sep 1 14:47:57 2022] GET /identity/v3/projects/7a2da326ed3543b197ba594c52ad0f0c => generated 361 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:47:57.610154 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e763aa-bf18-43d9-94ee-6fbf857c6a7a None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:47:57.709216 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e763aa-bf18-43d9-94ee-6fbf857c6a7a None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:47:57.720190 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:47:57.744077 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918e0dde-adee-4bd6-84e6-249120135e8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:47:57.744723 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918e0dde-adee-4bd6-84e6-249120135e8d None admin] REQUEST_METHOD: `PUT` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:47:57.745003 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918e0dde-adee-4bd6-84e6-249120135e8d None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:47:57.745275 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918e0dde-adee-4bd6-84e6-249120135e8d None admin] PATH_INFO: `/v3/projects/7a2da326ed3543b197ba594c52ad0f0c/users/c19f22bfb4e54b44b6775f3c6ee80fc6/roles/2a2adb769e55476b82e5014897e0a9c0` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:47:57.747187 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918e0dde-adee-4bd6-84e6-249120135e8d None admin] RBAC: Authorizing `identity:create_grant(project_id=7a2da326ed3543b197ba594c52ad0f0c, user_id=c19f22bfb4e54b44b6775f3c6ee80fc6, role_id=2a2adb769e55476b82e5014897e0a9c0)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:47:57.748634 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918e0dde-adee-4bd6-84e6-249120135e8d None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:47:57.760706 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 592/1192] 10.222.0.33 () {58 vars in 1439 bytes} [Thu Sep 1 14:47:57 2022] PUT /identity/v3/projects/7a2da326ed3543b197ba594c52ad0f0c/users/c19f22bfb4e54b44b6775f3c6ee80fc6/roles/2a2adb769e55476b82e5014897e0a9c0 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:47:59.625317 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c990aa-0de3-46ad-87d1-57ea78b23922 None None] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:47:59.625873 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c990aa-0de3-46ad-87d1-57ea78b23922 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:47:59.625873 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c990aa-0de3-46ad-87d1-57ea78b23922 None None] PATH_INFO: `/` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:47:59.627253 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 601/1193] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 1 14:47:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:47:59.632906 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53673aa8-978b-42a3-bdbe-56263011e4f6 None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:47:59.633319 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53673aa8-978b-42a3-bdbe-56263011e4f6 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:47:59.633451 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53673aa8-978b-42a3-bdbe-56263011e4f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:47:59.651799 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-53673aa8-978b-42a3-bdbe-56263011e4f6 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:47:59.653002 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-53673aa8-978b-42a3-bdbe-56263011e4f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:47:59.717690 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 593/1194] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 1 14:47:59 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 14:47:59.983193 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c990aa-0de3-46ad-87d1-57ea78b23922 None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:48:00.030748 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:48:00.084188 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c990aa-0de3-46ad-87d1-57ea78b23922 None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:48:00.116640 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1373bbd5-fe09-445d-bc62-8f5994df0785 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:48:00.117253 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1373bbd5-fe09-445d-bc62-8f5994df0785 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:48:00.117533 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1373bbd5-fe09-445d-bc62-8f5994df0785 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:48:00.117805 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1373bbd5-fe09-445d-bc62-8f5994df0785 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:48:00.127498 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1373bbd5-fe09-445d-bc62-8f5994df0785 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 14:48:00.132969 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application [None req-1373bbd5-fe09-445d-bc62-8f5994df0785 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 14:48:00.132969 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:48:00.132969 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:48:00.132969 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:48:00.132969 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:48:00.132969 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:48:00.132969 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:48:00.132969 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:48:00.132969 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:48:00.132969 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:48:00.132969 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:48:00.132969 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:48:00.132969 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 14:48:00.132969 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 14:48:00.132969 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 14:48:00.132969 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 14:48:00.134092 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 14:48:00.134092 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 14:48:00.134092 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 14:48:00.134092 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 14:48:00.134092 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:48:00.134092 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:48:00.134092 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:48:00.134092 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:48:00.134092 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:48:00.134092 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:48:00.134092 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:48:00.134092 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application with Lock( Sep 01 14:48:00.134092 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:48:00.134092 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:48:00.134092 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:48:00.134092 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:48:00.135209 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:48:00.135209 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:48:00.135209 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:48:00.135209 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:48:00.135209 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 14:48:00.135209 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 14:48:00.135209 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 14:48:00.135209 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 14:48:00.135209 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 14:48:00.135209 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 14:48:00.135209 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 14:48:00.135209 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Sep 01 14:48:00.135209 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 602/1195] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Sep 1 14:47:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:48:00.140150 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53673aa8-978b-42a3-bdbe-56263011e4f6 None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:48:00.232446 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:48:00.235424 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53673aa8-978b-42a3-bdbe-56263011e4f6 None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:48:00.269433 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f17bdc2-b0ac-4407-9f4b-7c986d5fc669 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:48:00.270047 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f17bdc2-b0ac-4407-9f4b-7c986d5fc669 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:48:00.270347 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f17bdc2-b0ac-4407-9f4b-7c986d5fc669 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:48:00.270620 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f17bdc2-b0ac-4407-9f4b-7c986d5fc669 None admin] PATH_INFO: `/v3/roles` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:48:00.271280 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f17bdc2-b0ac-4407-9f4b-7c986d5fc669 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81341) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:48:00.271578 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f17bdc2-b0ac-4407-9f4b-7c986d5fc669 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:48:00.272959 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f17bdc2-b0ac-4407-9f4b-7c986d5fc669 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:48:00.280679 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 594/1196] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Sep 1 14:48:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:48:00.286956 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1373bbd5-fe09-445d-bc62-8f5994df0785 None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:48:00.383150 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1373bbd5-fe09-445d-bc62-8f5994df0785 None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:48:00.390228 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:48:00.416479 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821bc4ab-1588-4005-898b-8cd46dc4b716 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:48:00.417089 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821bc4ab-1588-4005-898b-8cd46dc4b716 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:48:00.417368 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821bc4ab-1588-4005-898b-8cd46dc4b716 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:48:00.417638 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821bc4ab-1588-4005-898b-8cd46dc4b716 None admin] PATH_INFO: `/v3/users/c19f22bfb4e54b44b6775f3c6ee80fc6` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:48:00.418779 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821bc4ab-1588-4005-898b-8cd46dc4b716 None admin] RBAC: Authorizing `identity:get_user(user_id=c19f22bfb4e54b44b6775f3c6ee80fc6)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:48:00.420203 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821bc4ab-1588-4005-898b-8cd46dc4b716 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:48:00.421749 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 603/1197] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Sep 1 14:48:00 2022] GET /identity/v3/users/c19f22bfb4e54b44b6775f3c6ee80fc6 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:48:00.428069 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f17bdc2-b0ac-4407-9f4b-7c986d5fc669 None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:48:00.524093 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f17bdc2-b0ac-4407-9f4b-7c986d5fc669 None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:48:00.556642 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f190c95-c439-4c02-a3d5-ed042d58144c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:48:00.557233 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f190c95-c439-4c02-a3d5-ed042d58144c None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:48:00.557514 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f190c95-c439-4c02-a3d5-ed042d58144c None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:48:00.557787 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f190c95-c439-4c02-a3d5-ed042d58144c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:48:00.586008 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:48:00.622954 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f190c95-c439-4c02-a3d5-ed042d58144c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:48:00.624406 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f190c95-c439-4c02-a3d5-ed042d58144c None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:48:00.718563 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 595/1198] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Sep 1 14:48:00 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 14:48:00.725020 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821bc4ab-1588-4005-898b-8cd46dc4b716 None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:48:00.819984 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821bc4ab-1588-4005-898b-8cd46dc4b716 None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:48:00.852894 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd034d3d-e90b-4fb0-8c25-fec05daba828 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:48:00.853498 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd034d3d-e90b-4fb0-8c25-fec05daba828 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:48:00.853773 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd034d3d-e90b-4fb0-8c25-fec05daba828 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:48:00.854040 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd034d3d-e90b-4fb0-8c25-fec05daba828 None admin] PATH_INFO: `/v3/projects/7a2da326ed3543b197ba594c52ad0f0c` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:48:00.855164 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd034d3d-e90b-4fb0-8c25-fec05daba828 None admin] RBAC: Authorizing `identity:get_project(project_id=7a2da326ed3543b197ba594c52ad0f0c)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:48:00.856595 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd034d3d-e90b-4fb0-8c25-fec05daba828 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:48:00.858146 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 604/1199] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Sep 1 14:48:00 2022] GET /identity/v3/projects/7a2da326ed3543b197ba594c52ad0f0c => generated 361 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:48:00.863995 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f190c95-c439-4c02-a3d5-ed042d58144c None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:48:00.922226 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:48:00.959956 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f190c95-c439-4c02-a3d5-ed042d58144c None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:48:00.992560 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fec72a0-8b7d-433a-a172-4ca556aae637 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:48:00.993148 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fec72a0-8b7d-433a-a172-4ca556aae637 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:48:00.993429 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fec72a0-8b7d-433a-a172-4ca556aae637 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:48:00.993700 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fec72a0-8b7d-433a-a172-4ca556aae637 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:48:00.994447 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fec72a0-8b7d-433a-a172-4ca556aae637 None admin] RBAC: Adding query filter params (role.id=2a2adb769e55476b82e5014897e0a9c0, scope.project.id=7a2da326ed3543b197ba594c52ad0f0c, user.id=c19f22bfb4e54b44b6775f3c6ee80fc6) {{(pid=81341) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:48:00.994758 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fec72a0-8b7d-433a-a172-4ca556aae637 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:48:00.996189 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fec72a0-8b7d-433a-a172-4ca556aae637 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:48:01.001735 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 596/1200] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Sep 1 14:48:00 2022] GET /identity/v3/role_assignments?user.id=c19f22bfb4e54b44b6775f3c6ee80fc6&scope.project.id=7a2da326ed3543b197ba594c52ad0f0c&role.id=2a2adb769e55476b82e5014897e0a9c0 => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:48:10.848234 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=79 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:48:10.999328 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=37 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:48:55.995870 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2707588d-92c3-47ca-a5e0-922c6e3029c5 None None] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:48:55.996449 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2707588d-92c3-47ca-a5e0-922c6e3029c5 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:48:55.996538 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2707588d-92c3-47ca-a5e0-922c6e3029c5 None None] PATH_INFO: `/` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:48:55.998099 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 605/1201] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 1 14:48:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:48:56.004455 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f431f51-e2f0-40d9-b1c5-9c6de28451c0 None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:48:56.004943 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f431f51-e2f0-40d9-b1c5-9c6de28451c0 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:48:56.005044 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f431f51-e2f0-40d9-b1c5-9c6de28451c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:48:56.030966 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-4f431f51-e2f0-40d9-b1c5-9c6de28451c0 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:48:56.032178 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-4f431f51-e2f0-40d9-b1c5-9c6de28451c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:48:56.117210 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 597/1202] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 1 14:48:56 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 14:48:56.399297 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2707588d-92c3-47ca-a5e0-922c6e3029c5 None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:48:56.562136 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2707588d-92c3-47ca-a5e0-922c6e3029c5 None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:48:56.610973 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49023f73-fb00-4f8b-994e-f911a039f12c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:48:56.611899 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49023f73-fb00-4f8b-994e-f911a039f12c None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:48:56.612248 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49023f73-fb00-4f8b-994e-f911a039f12c None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:48:56.613314 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49023f73-fb00-4f8b-994e-f911a039f12c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:48:56.626752 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49023f73-fb00-4f8b-994e-f911a039f12c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:48:56.628588 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49023f73-fb00-4f8b-994e-f911a039f12c None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:48:56.650047 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application [None req-49023f73-fb00-4f8b-994e-f911a039f12c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 14:48:56.650047 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:48:56.650047 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:48:56.650047 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:48:56.650047 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:48:56.650047 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:48:56.650047 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:48:56.650047 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:48:56.650047 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:48:56.650047 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:48:56.650047 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:48:56.650047 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:48:56.650047 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 14:48:56.650047 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 14:48:56.650047 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 14:48:56.650047 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 14:48:56.654453 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:48:56.654453 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:48:56.654453 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:48:56.654453 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:48:56.654453 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:48:56.654453 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:48:56.654453 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:48:56.654453 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application with Lock( Sep 01 14:48:56.654453 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:48:56.654453 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:48:56.654453 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:48:56.654453 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:48:56.654453 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:48:56.654453 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:48:56.654453 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:48:56.654453 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:48:56.654453 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 14:48:56.655900 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 14:48:56.655900 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 14:48:56.655900 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Sep 01 14:48:56.655900 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 606/1203] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 1 14:48:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:48:56.658781 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f431f51-e2f0-40d9-b1c5-9c6de28451c0 None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:48:56.798889 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f431f51-e2f0-40d9-b1c5-9c6de28451c0 None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:48:56.836666 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:48:56.844676 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df7e513-399a-48d6-8ef4-707d8aaf8142 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:48:56.845361 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df7e513-399a-48d6-8ef4-707d8aaf8142 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:48:56.845641 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df7e513-399a-48d6-8ef4-707d8aaf8142 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:48:56.845909 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df7e513-399a-48d6-8ef4-707d8aaf8142 None admin] PATH_INFO: `/v3/domains` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:48:56.846650 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df7e513-399a-48d6-8ef4-707d8aaf8142 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81341) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:48:56.847662 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df7e513-399a-48d6-8ef4-707d8aaf8142 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:48:56.849376 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df7e513-399a-48d6-8ef4-707d8aaf8142 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:48:56.868310 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 598/1204] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Sep 1 14:48:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:48:56.873148 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49023f73-fb00-4f8b-994e-f911a039f12c None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:48:56.941589 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:48:57.004159 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49023f73-fb00-4f8b-994e-f911a039f12c None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:48:57.051187 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbda5583-a6ee-4e70-96bc-5c4249bfd74d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:48:57.052187 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbda5583-a6ee-4e70-96bc-5c4249bfd74d None admin] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:48:57.052694 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbda5583-a6ee-4e70-96bc-5c4249bfd74d None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:48:57.053190 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbda5583-a6ee-4e70-96bc-5c4249bfd74d None admin] PATH_INFO: `/v3/users` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:48:57.054046 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbda5583-a6ee-4e70-96bc-5c4249bfd74d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:48:57.055815 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbda5583-a6ee-4e70-96bc-5c4249bfd74d None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:48:57.094477 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 607/1205] 10.222.0.33 () {60 vars in 1132 bytes} [Thu Sep 1 14:48:56 2022] POST /identity/v3/users => generated 251 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:48:59.146087 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49802e76-f0c5-48ff-81fa-1c873db5c8ec None None] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:48:59.146667 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49802e76-f0c5-48ff-81fa-1c873db5c8ec None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:48:59.146772 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49802e76-f0c5-48ff-81fa-1c873db5c8ec None None] PATH_INFO: `/` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:48:59.148814 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 599/1206] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 1 14:48:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 14:48:59.155226 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e15fd3-01d3-46e0-ae09-7dc404d7f8db None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:48:59.155649 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e15fd3-01d3-46e0-ae09-7dc404d7f8db None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:48:59.156055 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e15fd3-01d3-46e0-ae09-7dc404d7f8db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:48:59.177152 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-f7e15fd3-01d3-46e0-ae09-7dc404d7f8db None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:48:59.178656 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-f7e15fd3-01d3-46e0-ae09-7dc404d7f8db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:48:59.256232 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 608/1207] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 1 14:48:59 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 14:48:59.562722 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49802e76-f0c5-48ff-81fa-1c873db5c8ec None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:48:59.688336 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49802e76-f0c5-48ff-81fa-1c873db5c8ec None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:48:59.734492 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9998911-5f9c-4798-abf7-a2f653d06a90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:48:59.736883 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9998911-5f9c-4798-abf7-a2f653d06a90 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:48:59.736883 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9998911-5f9c-4798-abf7-a2f653d06a90 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:48:59.736883 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9998911-5f9c-4798-abf7-a2f653d06a90 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:48:59.747963 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9998911-5f9c-4798-abf7-a2f653d06a90 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 14:48:59.753493 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application [None req-b9998911-5f9c-4798-abf7-a2f653d06a90 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 14:48:59.753493 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:48:59.753493 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:48:59.753493 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:48:59.753493 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:48:59.753493 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:48:59.753493 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:48:59.753493 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:48:59.753493 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:48:59.753493 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:48:59.753493 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:48:59.753493 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:48:59.753493 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 14:48:59.753493 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 14:48:59.753493 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 14:48:59.753493 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 14:48:59.758681 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 14:48:59.758681 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 14:48:59.758681 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 14:48:59.758681 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 14:48:59.758681 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:48:59.758681 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:48:59.758681 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:48:59.758681 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:48:59.758681 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:48:59.758681 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:48:59.758681 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:48:59.758681 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application with Lock( Sep 01 14:48:59.758681 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:48:59.758681 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:48:59.758681 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:48:59.758681 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:48:59.759751 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:48:59.759751 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:48:59.759751 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:48:59.759751 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:48:59.759751 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 14:48:59.759751 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 14:48:59.759751 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 14:48:59.759751 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 14:48:59.759751 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 14:48:59.759751 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 14:48:59.759751 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 14:48:59.759751 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Sep 01 14:48:59.759751 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 600/1208] 10.222.0.33 () {56 vars in 1102 bytes} [Thu Sep 1 14:48:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:48:59.761470 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e15fd3-01d3-46e0-ae09-7dc404d7f8db None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:48:59.806625 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:48:59.876346 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e15fd3-01d3-46e0-ae09-7dc404d7f8db None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:48:59.914993 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e323a72-02ed-449b-b31a-a697216d98e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:48:59.915693 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e323a72-02ed-449b-b31a-a697216d98e6 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:48:59.915930 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e323a72-02ed-449b-b31a-a697216d98e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:48:59.916294 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e323a72-02ed-449b-b31a-a697216d98e6 None admin] PATH_INFO: `/v3/roles` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:48:59.916953 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e323a72-02ed-449b-b31a-a697216d98e6 None admin] RBAC: Adding query filter params (name=service) {{(pid=81342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:48:59.917253 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e323a72-02ed-449b-b31a-a697216d98e6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:48:59.918850 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e323a72-02ed-449b-b31a-a697216d98e6 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:48:59.928455 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 609/1209] 10.222.0.33 () {56 vars in 1103 bytes} [Thu Sep 1 14:48:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:48:59.935449 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9998911-5f9c-4798-abf7-a2f653d06a90 None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:48:59.992558 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:49:00.057004 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9998911-5f9c-4798-abf7-a2f653d06a90 None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:49:00.102442 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc403bf-22bc-4b41-973d-f56c66451977 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:49:00.103072 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc403bf-22bc-4b41-973d-f56c66451977 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:49:00.103351 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc403bf-22bc-4b41-973d-f56c66451977 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:49:00.103624 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc403bf-22bc-4b41-973d-f56c66451977 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:49:00.115343 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc403bf-22bc-4b41-973d-f56c66451977 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:49:00.116811 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc403bf-22bc-4b41-973d-f56c66451977 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:49:00.134211 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application [None req-7bc403bf-22bc-4b41-973d-f56c66451977 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 14:49:00.134211 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:49:00.134211 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:49:00.134211 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:49:00.134211 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:49:00.134211 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:49:00.134211 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:49:00.134211 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:49:00.134211 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:49:00.134211 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:49:00.134211 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:49:00.134211 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:49:00.134211 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 14:49:00.134211 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 14:49:00.134211 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 14:49:00.134211 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 14:49:00.140067 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:49:00.140067 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:49:00.140067 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:49:00.140067 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:49:00.140067 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:49:00.140067 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:49:00.140067 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:49:00.140067 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application with Lock( Sep 01 14:49:00.140067 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:49:00.140067 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:49:00.140067 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:49:00.140067 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:49:00.140067 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:49:00.140067 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:49:00.140067 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:49:00.140067 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:49:00.140067 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 14:49:00.141203 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 14:49:00.141203 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 14:49:00.141203 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Sep 01 14:49:00.141203 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 601/1210] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 1 14:48:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:49:00.142979 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e323a72-02ed-449b-b31a-a697216d98e6 None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:49:00.256507 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:49:00.260077 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e323a72-02ed-449b-b31a-a697216d98e6 None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:49:00.295994 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfe40cf-e672-4cd4-a3b6-e0320cc42479 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:49:00.296764 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfe40cf-e672-4cd4-a3b6-e0320cc42479 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:49:00.297143 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfe40cf-e672-4cd4-a3b6-e0320cc42479 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:49:00.297563 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfe40cf-e672-4cd4-a3b6-e0320cc42479 None admin] PATH_INFO: `/v3/domains` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:49:00.298367 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfe40cf-e672-4cd4-a3b6-e0320cc42479 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:49:00.298726 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfe40cf-e672-4cd4-a3b6-e0320cc42479 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:49:00.300716 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfe40cf-e672-4cd4-a3b6-e0320cc42479 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:49:00.313079 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 610/1211] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Sep 1 14:49:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:49:00.319180 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc403bf-22bc-4b41-973d-f56c66451977 None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:49:00.440273 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc403bf-22bc-4b41-973d-f56c66451977 None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:49:00.442005 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:49:00.481835 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1545c6-991f-4711-b5b2-3307b88f03de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:49:00.483089 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1545c6-991f-4711-b5b2-3307b88f03de None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:49:00.483089 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1545c6-991f-4711-b5b2-3307b88f03de None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:49:00.483307 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1545c6-991f-4711-b5b2-3307b88f03de None admin] PATH_INFO: `/v3/users/placement` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:49:00.493479 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1545c6-991f-4711-b5b2-3307b88f03de None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:49:00.495341 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1545c6-991f-4711-b5b2-3307b88f03de None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:49:00.508459 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application [None req-ea1545c6-991f-4711-b5b2-3307b88f03de None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 01 14:49:00.508459 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:49:00.508459 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:49:00.508459 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:49:00.508459 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:49:00.508459 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:49:00.508459 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:49:00.508459 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:49:00.508459 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:49:00.508459 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:49:00.508459 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:49:00.508459 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:49:00.508459 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 14:49:00.508459 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 14:49:00.508459 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 14:49:00.508459 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 14:49:00.509868 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:49:00.509868 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:49:00.509868 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 14:49:00.509868 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 14:49:00.509868 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 14:49:00.509868 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 14:49:00.509868 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:49:00.509868 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:49:00.509868 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:49:00.509868 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:49:00.509868 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:49:00.509868 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application with Lock( Sep 01 14:49:00.509868 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:49:00.509868 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:49:00.509868 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:49:00.509868 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:49:00.509868 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:49:00.509868 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:49:00.510959 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:49:00.510959 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:49:00.510959 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 14:49:00.510959 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 14:49:00.510959 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 14:49:00.510959 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 14:49:00.510959 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 14:49:00.510959 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 14:49:00.510959 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 01 14:49:00.510959 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Sep 01 14:49:00.510959 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 602/1212] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 1 14:49:00 2022] GET /identity/v3/users/placement => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:49:00.515728 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfe40cf-e672-4cd4-a3b6-e0320cc42479 None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:49:00.642569 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfe40cf-e672-4cd4-a3b6-e0320cc42479 None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:49:00.682473 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ecfe5a-8d39-4e8b-8040-3e3c88b15a72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:49:00.683237 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ecfe5a-8d39-4e8b-8040-3e3c88b15a72 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:49:00.683613 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ecfe5a-8d39-4e8b-8040-3e3c88b15a72 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:49:00.683981 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ecfe5a-8d39-4e8b-8040-3e3c88b15a72 None admin] PATH_INFO: `/v3/users` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:49:00.684828 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ecfe5a-8d39-4e8b-8040-3e3c88b15a72 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:49:00.685202 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ecfe5a-8d39-4e8b-8040-3e3c88b15a72 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:49:00.686665 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ecfe5a-8d39-4e8b-8040-3e3c88b15a72 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:49:00.706849 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 611/1213] 10.222.0.33 () {56 vars in 1143 bytes} [Thu Sep 1 14:49:00 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:49:00.712729 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1545c6-991f-4711-b5b2-3307b88f03de None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:49:00.836511 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1545c6-991f-4711-b5b2-3307b88f03de None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:49:00.883513 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ae2128-d1f4-4f55-9815-25fe2c25736c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:49:00.884147 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ae2128-d1f4-4f55-9815-25fe2c25736c None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:49:00.884466 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ae2128-d1f4-4f55-9815-25fe2c25736c None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:49:00.884712 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ae2128-d1f4-4f55-9815-25fe2c25736c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:49:00.918749 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:49:00.965118 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ae2128-d1f4-4f55-9815-25fe2c25736c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:49:00.966626 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ae2128-d1f4-4f55-9815-25fe2c25736c None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:49:01.085040 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 603/1214] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Sep 1 14:49:00 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 14:49:01.091120 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ecfe5a-8d39-4e8b-8040-3e3c88b15a72 None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:49:01.111932 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:49:01.204812 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ecfe5a-8d39-4e8b-8040-3e3c88b15a72 None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:49:01.248029 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65528c74-69a6-4948-a9ef-3733d02a13ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:49:01.248716 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65528c74-69a6-4948-a9ef-3733d02a13ea None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:49:01.249104 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65528c74-69a6-4948-a9ef-3733d02a13ea None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:49:01.249471 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65528c74-69a6-4948-a9ef-3733d02a13ea None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:49:01.261025 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65528c74-69a6-4948-a9ef-3733d02a13ea None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:49:01.262920 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65528c74-69a6-4948-a9ef-3733d02a13ea None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:49:01.280412 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application [None req-65528c74-69a6-4948-a9ef-3733d02a13ea None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 14:49:01.280412 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:49:01.280412 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:49:01.280412 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:49:01.280412 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:49:01.280412 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:49:01.280412 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:49:01.280412 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:49:01.280412 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:49:01.280412 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:49:01.280412 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:49:01.280412 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:49:01.280412 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 14:49:01.280412 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 14:49:01.280412 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 14:49:01.280412 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 14:49:01.281666 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:49:01.281666 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:49:01.281666 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:49:01.281666 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:49:01.281666 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:49:01.281666 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:49:01.281666 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:49:01.281666 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application with Lock( Sep 01 14:49:01.281666 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:49:01.281666 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:49:01.281666 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:49:01.281666 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:49:01.281666 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:49:01.281666 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:49:01.281666 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:49:01.281666 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:49:01.281666 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 14:49:01.282800 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 14:49:01.282800 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 14:49:01.282800 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Sep 01 14:49:01.284745 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 612/1215] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 1 14:49:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:49:01.290550 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ae2128-d1f4-4f55-9815-25fe2c25736c None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:49:01.361134 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:49:01.402462 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ae2128-d1f4-4f55-9815-25fe2c25736c None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:49:01.439235 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78447f77-e14a-49d6-a1b9-55afc457559a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:49:01.439806 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78447f77-e14a-49d6-a1b9-55afc457559a None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:49:01.440097 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78447f77-e14a-49d6-a1b9-55afc457559a None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:49:01.440473 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78447f77-e14a-49d6-a1b9-55afc457559a None admin] PATH_INFO: `/v3/domains` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:49:01.441164 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78447f77-e14a-49d6-a1b9-55afc457559a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81341) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:49:01.441470 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78447f77-e14a-49d6-a1b9-55afc457559a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:49:01.443178 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78447f77-e14a-49d6-a1b9-55afc457559a None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:49:01.455649 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 604/1216] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Sep 1 14:49:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:49:01.461842 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65528c74-69a6-4948-a9ef-3733d02a13ea None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:49:01.541876 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:49:01.577153 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65528c74-69a6-4948-a9ef-3733d02a13ea None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:49:01.613176 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b269e8-1beb-4a28-9833-c386f78fe2a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:49:01.613813 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b269e8-1beb-4a28-9833-c386f78fe2a0 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:49:01.614099 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b269e8-1beb-4a28-9833-c386f78fe2a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:49:01.614443 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b269e8-1beb-4a28-9833-c386f78fe2a0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:49:01.621556 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b269e8-1beb-4a28-9833-c386f78fe2a0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:49:01.623065 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b269e8-1beb-4a28-9833-c386f78fe2a0 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:49:01.635953 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application [None req-25b269e8-1beb-4a28-9833-c386f78fe2a0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 14:49:01.635953 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:49:01.635953 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:49:01.635953 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:49:01.635953 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:49:01.635953 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:49:01.635953 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:49:01.635953 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:49:01.635953 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:49:01.635953 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:49:01.635953 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:49:01.635953 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:49:01.635953 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 14:49:01.635953 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 14:49:01.635953 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 14:49:01.637163 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 14:49:01.637163 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:49:01.637163 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:49:01.637163 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:49:01.637163 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:49:01.637163 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:49:01.637163 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:49:01.637163 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:49:01.637163 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application with Lock( Sep 01 14:49:01.637163 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:49:01.637163 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:49:01.637163 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:49:01.637163 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:49:01.637163 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:49:01.637163 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:49:01.637163 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:49:01.637163 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:49:01.638672 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 14:49:01.638672 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 14:49:01.638672 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 14:49:01.638672 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 14:49:01.638672 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 14:49:01.638672 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 14:49:01.638672 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 14:49:01.638672 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Sep 01 14:49:01.638672 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 613/1217] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Sep 1 14:49:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:49:01.644212 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78447f77-e14a-49d6-a1b9-55afc457559a None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:49:01.766120 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78447f77-e14a-49d6-a1b9-55afc457559a None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:49:01.792052 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:49:01.806097 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736a4f87-d248-419b-8564-6b4987139ac5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:49:01.806792 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736a4f87-d248-419b-8564-6b4987139ac5 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:49:01.807143 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736a4f87-d248-419b-8564-6b4987139ac5 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:49:01.807478 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736a4f87-d248-419b-8564-6b4987139ac5 None admin] PATH_INFO: `/v3/projects` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:49:01.808444 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736a4f87-d248-419b-8564-6b4987139ac5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81341) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:49:01.808817 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736a4f87-d248-419b-8564-6b4987139ac5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:49:01.810458 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736a4f87-d248-419b-8564-6b4987139ac5 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:49:01.825571 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 605/1218] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Sep 1 14:49:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:49:01.832270 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b269e8-1beb-4a28-9833-c386f78fe2a0 None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:49:01.957514 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b269e8-1beb-4a28-9833-c386f78fe2a0 None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:49:01.996017 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:49:02.003631 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a586394b-12e4-4344-9a95-93e14e46b6dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:49:02.004995 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a586394b-12e4-4344-9a95-93e14e46b6dd None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:49:02.004995 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a586394b-12e4-4344-9a95-93e14e46b6dd None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:49:02.004995 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a586394b-12e4-4344-9a95-93e14e46b6dd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:49:02.005790 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a586394b-12e4-4344-9a95-93e14e46b6dd None admin] RBAC: Adding query filter params (role.id=5e3f992e804046f381b3de8bf2a512e2, scope.project.id=e8c85b4bd17e4e89b56bf0f831b474d5, user.id=7f312c8dcabe48ef9ea453364ade38ff) {{(pid=81342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:49:02.006120 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a586394b-12e4-4344-9a95-93e14e46b6dd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:49:02.007620 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a586394b-12e4-4344-9a95-93e14e46b6dd None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:49:02.015301 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 614/1219] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Sep 1 14:49:01 2022] GET /identity/v3/role_assignments?user.id=7f312c8dcabe48ef9ea453364ade38ff&scope.project.id=e8c85b4bd17e4e89b56bf0f831b474d5&role.id=5e3f992e804046f381b3de8bf2a512e2 => generated 259 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:49:04.125604 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c6e642-c146-41a2-948c-138b9cb86329 None None] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:49:04.126170 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c6e642-c146-41a2-948c-138b9cb86329 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:49:04.126170 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c6e642-c146-41a2-948c-138b9cb86329 None None] PATH_INFO: `/` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:49:04.127561 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 606/1220] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 1 14:49:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:49:04.133902 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e22cd97-f34a-4d2c-84ea-4e314c75111c None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:49:04.134355 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e22cd97-f34a-4d2c-84ea-4e314c75111c None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:49:04.134463 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e22cd97-f34a-4d2c-84ea-4e314c75111c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:49:04.154360 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-8e22cd97-f34a-4d2c-84ea-4e314c75111c None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:49:04.155559 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-8e22cd97-f34a-4d2c-84ea-4e314c75111c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:49:04.224565 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 615/1221] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 1 14:49:04 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 14:49:04.491978 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c6e642-c146-41a2-948c-138b9cb86329 None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:49:04.604142 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c6e642-c146-41a2-948c-138b9cb86329 None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:49:04.641631 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d40c96c-e3d7-41dd-ba7b-286e5f465348 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:49:04.642349 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d40c96c-e3d7-41dd-ba7b-286e5f465348 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:49:04.642561 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d40c96c-e3d7-41dd-ba7b-286e5f465348 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:49:04.642833 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d40c96c-e3d7-41dd-ba7b-286e5f465348 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:49:04.657627 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d40c96c-e3d7-41dd-ba7b-286e5f465348 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 14:49:04.663924 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application [None req-5d40c96c-e3d7-41dd-ba7b-286e5f465348 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 14:49:04.663924 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:49:04.663924 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:49:04.663924 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:49:04.663924 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:49:04.663924 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:49:04.663924 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:49:04.663924 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:49:04.663924 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:49:04.663924 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:49:04.663924 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:49:04.663924 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:49:04.663924 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 14:49:04.663924 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 14:49:04.663924 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 14:49:04.663924 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 14:49:04.665041 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 14:49:04.665041 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 14:49:04.665041 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 14:49:04.665041 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 14:49:04.665041 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:49:04.665041 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:49:04.665041 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:49:04.665041 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:49:04.665041 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:49:04.665041 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:49:04.665041 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:49:04.665041 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application with Lock( Sep 01 14:49:04.665041 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:49:04.665041 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:49:04.665041 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:49:04.665041 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:49:04.666095 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:49:04.666095 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:49:04.666095 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:49:04.666095 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:49:04.666095 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 14:49:04.666095 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 14:49:04.666095 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 14:49:04.666095 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 14:49:04.666095 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 14:49:04.666095 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 14:49:04.666095 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 14:49:04.666095 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Sep 01 14:49:04.666095 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 607/1222] 10.222.0.33 () {56 vars in 1102 bytes} [Thu Sep 1 14:49:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:49:04.671359 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e22cd97-f34a-4d2c-84ea-4e314c75111c None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:49:04.791673 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e22cd97-f34a-4d2c-84ea-4e314c75111c None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:49:04.818063 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:49:04.833825 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d83fdc-b74b-4953-8a33-2526d936343b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:49:04.834562 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d83fdc-b74b-4953-8a33-2526d936343b None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:49:04.834874 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d83fdc-b74b-4953-8a33-2526d936343b None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:49:04.835264 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d83fdc-b74b-4953-8a33-2526d936343b None admin] PATH_INFO: `/v3/roles` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:49:04.836124 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d83fdc-b74b-4953-8a33-2526d936343b None admin] RBAC: Adding query filter params (name=service) {{(pid=81342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:49:04.836422 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d83fdc-b74b-4953-8a33-2526d936343b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:49:04.837783 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d83fdc-b74b-4953-8a33-2526d936343b None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:49:04.846251 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 616/1223] 10.222.0.33 () {56 vars in 1103 bytes} [Thu Sep 1 14:49:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 14:49:04.852351 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d40c96c-e3d7-41dd-ba7b-286e5f465348 None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:49:04.900817 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:49:04.966639 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d40c96c-e3d7-41dd-ba7b-286e5f465348 None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:49:05.000329 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffaee5dd-6bda-4ae9-ace9-9dad8753c4cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:49:05.000981 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffaee5dd-6bda-4ae9-ace9-9dad8753c4cc None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:49:05.001266 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffaee5dd-6bda-4ae9-ace9-9dad8753c4cc None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:49:05.001544 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffaee5dd-6bda-4ae9-ace9-9dad8753c4cc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:49:05.014744 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffaee5dd-6bda-4ae9-ace9-9dad8753c4cc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:49:05.016229 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffaee5dd-6bda-4ae9-ace9-9dad8753c4cc None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:49:05.031388 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application [None req-ffaee5dd-6bda-4ae9-ace9-9dad8753c4cc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 14:49:05.031388 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:49:05.031388 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:49:05.031388 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:49:05.031388 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:49:05.031388 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:49:05.031388 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:49:05.031388 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:49:05.031388 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:49:05.031388 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:49:05.031388 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:49:05.031388 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:49:05.031388 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 14:49:05.031388 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 14:49:05.031388 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 14:49:05.031388 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 14:49:05.032433 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:49:05.032433 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:49:05.032433 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:49:05.032433 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:49:05.032433 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:49:05.032433 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:49:05.032433 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:49:05.032433 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application with Lock( Sep 01 14:49:05.032433 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:49:05.032433 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:49:05.032433 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:49:05.032433 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:49:05.032433 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:49:05.032433 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:49:05.032433 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:49:05.032433 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:49:05.032433 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 14:49:05.033665 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 14:49:05.033665 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 14:49:05.033665 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Sep 01 14:49:05.033665 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 608/1224] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 1 14:49:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:49:05.038721 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d83fdc-b74b-4953-8a33-2526d936343b None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:49:05.148806 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d83fdc-b74b-4953-8a33-2526d936343b None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:49:05.184329 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fbfbbe-1cda-448d-8e8c-33a6e3f0a8be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:49:05.184941 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fbfbbe-1cda-448d-8e8c-33a6e3f0a8be None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:49:05.185218 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fbfbbe-1cda-448d-8e8c-33a6e3f0a8be None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:49:05.185488 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fbfbbe-1cda-448d-8e8c-33a6e3f0a8be None admin] PATH_INFO: `/v3/domains` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:49:05.186137 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fbfbbe-1cda-448d-8e8c-33a6e3f0a8be None admin] RBAC: Adding query filter params (name=Default) {{(pid=81342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:49:05.186490 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fbfbbe-1cda-448d-8e8c-33a6e3f0a8be None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:49:05.187843 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fbfbbe-1cda-448d-8e8c-33a6e3f0a8be None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:49:05.200803 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 617/1225] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Sep 1 14:49:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 14:49:05.206612 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffaee5dd-6bda-4ae9-ace9-9dad8753c4cc None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:49:05.301855 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:49:05.309987 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffaee5dd-6bda-4ae9-ace9-9dad8753c4cc None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:49:05.346662 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d66e81-8f9f-41e9-81ff-3a43ff338eab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:49:05.347290 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d66e81-8f9f-41e9-81ff-3a43ff338eab None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:49:05.347567 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d66e81-8f9f-41e9-81ff-3a43ff338eab None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:49:05.347836 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d66e81-8f9f-41e9-81ff-3a43ff338eab None admin] PATH_INFO: `/v3/users/placement` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:49:05.356063 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d66e81-8f9f-41e9-81ff-3a43ff338eab None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:49:05.357528 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d66e81-8f9f-41e9-81ff-3a43ff338eab None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:49:05.369416 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application [None req-85d66e81-8f9f-41e9-81ff-3a43ff338eab None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 01 14:49:05.369416 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:49:05.369416 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:49:05.369416 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:49:05.369416 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:49:05.369416 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:49:05.369416 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:49:05.369416 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:49:05.369416 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:49:05.369416 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:49:05.369416 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:49:05.369416 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:49:05.369416 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 14:49:05.369416 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 14:49:05.369416 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 14:49:05.369416 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 14:49:05.370609 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:49:05.370609 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:49:05.370609 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 14:49:05.370609 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 14:49:05.370609 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 14:49:05.370609 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 14:49:05.370609 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:49:05.370609 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:49:05.370609 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:49:05.370609 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:49:05.370609 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:49:05.370609 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application with Lock( Sep 01 14:49:05.370609 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:49:05.370609 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:49:05.370609 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:49:05.370609 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:49:05.370609 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:49:05.370609 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:49:05.372220 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:49:05.372220 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:49:05.372220 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 14:49:05.372220 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 14:49:05.372220 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 14:49:05.372220 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 14:49:05.372220 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 14:49:05.372220 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 14:49:05.372220 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 01 14:49:05.372220 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Sep 01 14:49:05.372220 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 609/1226] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 1 14:49:05 2022] GET /identity/v3/users/placement => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:49:05.376515 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fbfbbe-1cda-448d-8e8c-33a6e3f0a8be None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:49:05.425717 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:49:05.483844 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fbfbbe-1cda-448d-8e8c-33a6e3f0a8be None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:49:05.527706 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597e9575-559c-4334-8154-3d30b01892d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:49:05.528346 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597e9575-559c-4334-8154-3d30b01892d0 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:49:05.528588 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597e9575-559c-4334-8154-3d30b01892d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:49:05.528879 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597e9575-559c-4334-8154-3d30b01892d0 None admin] PATH_INFO: `/v3/users` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:49:05.529618 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597e9575-559c-4334-8154-3d30b01892d0 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:49:05.529945 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597e9575-559c-4334-8154-3d30b01892d0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:49:05.531383 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597e9575-559c-4334-8154-3d30b01892d0 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:49:05.549036 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 618/1227] 10.222.0.33 () {56 vars in 1143 bytes} [Thu Sep 1 14:49:05 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:49:05.555492 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d66e81-8f9f-41e9-81ff-3a43ff338eab None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:49:05.669212 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d66e81-8f9f-41e9-81ff-3a43ff338eab None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:49:05.706164 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeac02a2-b6ce-462e-851a-cc31080db8bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:49:05.706741 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeac02a2-b6ce-462e-851a-cc31080db8bd None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:49:05.707023 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeac02a2-b6ce-462e-851a-cc31080db8bd None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:49:05.707294 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeac02a2-b6ce-462e-851a-cc31080db8bd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:49:05.719597 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeac02a2-b6ce-462e-851a-cc31080db8bd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:49:05.721314 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeac02a2-b6ce-462e-851a-cc31080db8bd None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:49:05.737408 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application [None req-aeac02a2-b6ce-462e-851a-cc31080db8bd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 14:49:05.737408 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:49:05.737408 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:49:05.737408 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:49:05.737408 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:49:05.737408 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:49:05.737408 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:49:05.737408 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:49:05.737408 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:49:05.737408 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:49:05.737408 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:49:05.737408 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:49:05.737408 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 14:49:05.737408 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 14:49:05.737408 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 14:49:05.737408 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 14:49:05.738667 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:49:05.738667 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:49:05.738667 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:49:05.738667 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:49:05.738667 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:49:05.738667 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:49:05.738667 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:49:05.738667 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application with Lock( Sep 01 14:49:05.738667 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:49:05.738667 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:49:05.738667 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:49:05.738667 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:49:05.738667 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:49:05.738667 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:49:05.738667 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:49:05.738667 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:49:05.738667 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 14:49:05.740312 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 14:49:05.740312 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 14:49:05.740312 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Sep 01 14:49:05.740312 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 610/1228] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 1 14:49:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:49:05.744996 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597e9575-559c-4334-8154-3d30b01892d0 None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:49:05.857774 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597e9575-559c-4334-8154-3d30b01892d0 None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:49:05.861164 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:49:05.896909 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcca934e-9b37-464a-af0d-38a4a42a9aa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:49:05.897599 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcca934e-9b37-464a-af0d-38a4a42a9aa4 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:49:05.897874 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcca934e-9b37-464a-af0d-38a4a42a9aa4 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:49:05.898138 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcca934e-9b37-464a-af0d-38a4a42a9aa4 None admin] PATH_INFO: `/v3/domains` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:49:05.898805 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcca934e-9b37-464a-af0d-38a4a42a9aa4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:49:05.899100 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcca934e-9b37-464a-af0d-38a4a42a9aa4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:49:05.900689 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcca934e-9b37-464a-af0d-38a4a42a9aa4 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:49:05.913532 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 619/1229] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Sep 1 14:49:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:49:05.919375 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeac02a2-b6ce-462e-851a-cc31080db8bd None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:49:05.928616 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:49:06.034786 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeac02a2-b6ce-462e-851a-cc31080db8bd None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:49:06.073957 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf6fccd-bba5-4f4d-9f0e-8a38fec4e6b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:49:06.074723 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf6fccd-bba5-4f4d-9f0e-8a38fec4e6b4 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:49:06.074936 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf6fccd-bba5-4f4d-9f0e-8a38fec4e6b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:49:06.075251 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf6fccd-bba5-4f4d-9f0e-8a38fec4e6b4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:49:06.083579 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf6fccd-bba5-4f4d-9f0e-8a38fec4e6b4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:49:06.085176 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf6fccd-bba5-4f4d-9f0e-8a38fec4e6b4 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:49:06.098255 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application [None req-5bf6fccd-bba5-4f4d-9f0e-8a38fec4e6b4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 14:49:06.098255 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:49:06.098255 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:49:06.098255 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:49:06.098255 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:49:06.098255 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:49:06.098255 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:49:06.098255 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:49:06.098255 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:49:06.098255 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:49:06.098255 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:49:06.098255 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:49:06.098255 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 14:49:06.098255 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 14:49:06.098255 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 14:49:06.099430 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 14:49:06.099430 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:49:06.099430 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:49:06.099430 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:49:06.099430 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:49:06.099430 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:49:06.099430 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:49:06.099430 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:49:06.099430 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application with Lock( Sep 01 14:49:06.099430 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:49:06.099430 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:49:06.099430 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:49:06.099430 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:49:06.099430 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:49:06.099430 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:49:06.099430 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:49:06.099430 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:49:06.100549 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 14:49:06.100549 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 14:49:06.100549 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 14:49:06.100549 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 14:49:06.100549 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 14:49:06.100549 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 14:49:06.100549 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 14:49:06.100549 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Sep 01 14:49:06.105025 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 611/1230] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Sep 1 14:49:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:49:06.105899 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcca934e-9b37-464a-af0d-38a4a42a9aa4 None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:49:06.210113 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcca934e-9b37-464a-af0d-38a4a42a9aa4 None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:49:06.249123 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f6b593-9a8d-47b3-85d0-dcd89f0a6d56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:49:06.249851 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f6b593-9a8d-47b3-85d0-dcd89f0a6d56 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:49:06.250126 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f6b593-9a8d-47b3-85d0-dcd89f0a6d56 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:49:06.250421 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f6b593-9a8d-47b3-85d0-dcd89f0a6d56 None admin] PATH_INFO: `/v3/projects` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:49:06.251240 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f6b593-9a8d-47b3-85d0-dcd89f0a6d56 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:49:06.251676 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f6b593-9a8d-47b3-85d0-dcd89f0a6d56 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:49:06.254531 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f6b593-9a8d-47b3-85d0-dcd89f0a6d56 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:49:06.270484 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 620/1231] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Sep 1 14:49:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 14:49:06.278227 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf6fccd-bba5-4f4d-9f0e-8a38fec4e6b4 None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:49:06.351986 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:49:06.395382 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf6fccd-bba5-4f4d-9f0e-8a38fec4e6b4 None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:49:06.431846 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110bcda1-3cd7-4dd0-b6b7-84015e7f6c86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:49:06.432494 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110bcda1-3cd7-4dd0-b6b7-84015e7f6c86 None admin] REQUEST_METHOD: `PUT` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:49:06.432766 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110bcda1-3cd7-4dd0-b6b7-84015e7f6c86 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:49:06.433035 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110bcda1-3cd7-4dd0-b6b7-84015e7f6c86 None admin] PATH_INFO: `/v3/projects/e8c85b4bd17e4e89b56bf0f831b474d5/users/7f312c8dcabe48ef9ea453364ade38ff/roles/5e3f992e804046f381b3de8bf2a512e2` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:49:06.449793 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110bcda1-3cd7-4dd0-b6b7-84015e7f6c86 None admin] RBAC: Authorizing `identity:create_grant(project_id=e8c85b4bd17e4e89b56bf0f831b474d5, user_id=7f312c8dcabe48ef9ea453364ade38ff, role_id=5e3f992e804046f381b3de8bf2a512e2)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:49:06.451307 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110bcda1-3cd7-4dd0-b6b7-84015e7f6c86 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:49:06.464795 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 612/1232] 10.222.0.33 () {58 vars in 1439 bytes} [Thu Sep 1 14:49:06 2022] PUT /identity/v3/projects/e8c85b4bd17e4e89b56bf0f831b474d5/users/7f312c8dcabe48ef9ea453364ade38ff/roles/5e3f992e804046f381b3de8bf2a512e2 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:49:08.454032 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231e3367-90bd-4bd4-bff2-2c848fb8eaee None None] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:49:08.454657 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231e3367-90bd-4bd4-bff2-2c848fb8eaee None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:49:08.454657 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231e3367-90bd-4bd4-bff2-2c848fb8eaee None None] PATH_INFO: `/` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:49:08.456014 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 621/1233] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 1 14:49:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:49:08.462665 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bda95f7-7768-4b60-b248-8ef1d8c02ea0 None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:49:08.463125 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bda95f7-7768-4b60-b248-8ef1d8c02ea0 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:49:08.463225 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bda95f7-7768-4b60-b248-8ef1d8c02ea0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:49:08.484187 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-9bda95f7-7768-4b60-b248-8ef1d8c02ea0 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:49:08.485528 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-9bda95f7-7768-4b60-b248-8ef1d8c02ea0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:49:08.560034 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 613/1234] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 1 14:49:08 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 14:49:08.863543 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231e3367-90bd-4bd4-bff2-2c848fb8eaee None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:49:08.894497 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:49:08.983429 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231e3367-90bd-4bd4-bff2-2c848fb8eaee None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:49:09.029852 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98f2d84-41ba-44e4-bbf8-72995a818106 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:49:09.030653 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98f2d84-41ba-44e4-bbf8-72995a818106 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:49:09.031017 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98f2d84-41ba-44e4-bbf8-72995a818106 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:49:09.031315 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98f2d84-41ba-44e4-bbf8-72995a818106 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:49:09.045308 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98f2d84-41ba-44e4-bbf8-72995a818106 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 14:49:09.051777 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application [None req-c98f2d84-41ba-44e4-bbf8-72995a818106 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 14:49:09.051777 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:49:09.051777 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:49:09.051777 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:49:09.051777 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:49:09.051777 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:49:09.051777 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:49:09.051777 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:49:09.051777 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:49:09.051777 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:49:09.051777 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:49:09.051777 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:49:09.051777 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 14:49:09.051777 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 14:49:09.051777 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 14:49:09.051777 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 14:49:09.052900 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 14:49:09.052900 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 14:49:09.052900 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 14:49:09.052900 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 14:49:09.052900 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:49:09.052900 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:49:09.052900 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:49:09.052900 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:49:09.052900 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:49:09.052900 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:49:09.052900 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:49:09.052900 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application with Lock( Sep 01 14:49:09.052900 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:49:09.052900 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:49:09.052900 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:49:09.052900 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:49:09.054132 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:49:09.054132 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:49:09.054132 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:49:09.054132 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:49:09.054132 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 14:49:09.054132 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 14:49:09.054132 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 14:49:09.054132 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 14:49:09.054132 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 14:49:09.054132 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 14:49:09.054132 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 14:49:09.054132 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Sep 01 14:49:09.054132 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 622/1235] 10.222.0.33 () {56 vars in 1102 bytes} [Thu Sep 1 14:49:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 14:49:09.061412 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bda95f7-7768-4b60-b248-8ef1d8c02ea0 None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:49:09.115386 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:49:09.172704 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bda95f7-7768-4b60-b248-8ef1d8c02ea0 None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:49:09.207807 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8590fd83-c7de-480f-9b6d-c3d8f4dc0b8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:49:09.208436 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8590fd83-c7de-480f-9b6d-c3d8f4dc0b8d None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:49:09.208701 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8590fd83-c7de-480f-9b6d-c3d8f4dc0b8d None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:49:09.208972 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8590fd83-c7de-480f-9b6d-c3d8f4dc0b8d None admin] PATH_INFO: `/v3/roles` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:49:09.209638 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8590fd83-c7de-480f-9b6d-c3d8f4dc0b8d None admin] RBAC: Adding query filter params (name=service) {{(pid=81341) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:49:09.209941 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8590fd83-c7de-480f-9b6d-c3d8f4dc0b8d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:49:09.211655 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8590fd83-c7de-480f-9b6d-c3d8f4dc0b8d None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:49:09.226392 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 614/1236] 10.222.0.33 () {56 vars in 1103 bytes} [Thu Sep 1 14:49:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 14:49:09.227622 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98f2d84-41ba-44e4-bbf8-72995a818106 None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:49:09.322345 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:49:09.340944 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98f2d84-41ba-44e4-bbf8-72995a818106 None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:49:09.376698 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30722f01-4cbf-468b-82b1-8cc3acc30b7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:49:09.377495 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30722f01-4cbf-468b-82b1-8cc3acc30b7f None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:49:09.379483 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30722f01-4cbf-468b-82b1-8cc3acc30b7f None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:49:09.379766 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30722f01-4cbf-468b-82b1-8cc3acc30b7f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:49:09.391794 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30722f01-4cbf-468b-82b1-8cc3acc30b7f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:49:09.393967 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30722f01-4cbf-468b-82b1-8cc3acc30b7f None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:49:09.410110 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application [None req-30722f01-4cbf-468b-82b1-8cc3acc30b7f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 14:49:09.410110 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:49:09.410110 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:49:09.410110 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:49:09.410110 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:49:09.410110 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:49:09.410110 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:49:09.410110 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:49:09.410110 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:49:09.410110 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:49:09.410110 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:49:09.410110 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:49:09.410110 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 14:49:09.410110 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 14:49:09.410110 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 14:49:09.410110 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 14:49:09.411310 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:49:09.411310 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:49:09.411310 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:49:09.411310 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:49:09.411310 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:49:09.411310 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:49:09.411310 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:49:09.411310 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application with Lock( Sep 01 14:49:09.411310 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:49:09.411310 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:49:09.411310 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:49:09.411310 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:49:09.411310 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:49:09.411310 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:49:09.411310 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:49:09.411310 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:49:09.411310 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 14:49:09.417227 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 14:49:09.417227 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 14:49:09.417227 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Sep 01 14:49:09.417227 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 623/1237] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 1 14:49:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 14:49:09.417531 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8590fd83-c7de-480f-9b6d-c3d8f4dc0b8d None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:49:09.525721 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8590fd83-c7de-480f-9b6d-c3d8f4dc0b8d None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:49:09.536550 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:49:09.566540 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc329be6-2f31-45a6-afa9-a3c63d33fc4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:49:09.567157 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc329be6-2f31-45a6-afa9-a3c63d33fc4c None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:49:09.567442 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc329be6-2f31-45a6-afa9-a3c63d33fc4c None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:49:09.567717 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc329be6-2f31-45a6-afa9-a3c63d33fc4c None admin] PATH_INFO: `/v3/domains` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:49:09.569405 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc329be6-2f31-45a6-afa9-a3c63d33fc4c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81341) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:49:09.569718 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc329be6-2f31-45a6-afa9-a3c63d33fc4c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:49:09.571498 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc329be6-2f31-45a6-afa9-a3c63d33fc4c None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:49:09.588258 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 615/1238] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Sep 1 14:49:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:49:09.594196 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30722f01-4cbf-468b-82b1-8cc3acc30b7f None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:49:09.704065 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30722f01-4cbf-468b-82b1-8cc3acc30b7f None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:49:09.738797 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665bb449-49c8-4c1c-87f7-9fdbf99c95f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:49:09.739417 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665bb449-49c8-4c1c-87f7-9fdbf99c95f7 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:49:09.739690 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665bb449-49c8-4c1c-87f7-9fdbf99c95f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:49:09.739957 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665bb449-49c8-4c1c-87f7-9fdbf99c95f7 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:49:09.742616 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:49:09.748021 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665bb449-49c8-4c1c-87f7-9fdbf99c95f7 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:49:09.749588 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665bb449-49c8-4c1c-87f7-9fdbf99c95f7 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:49:09.761296 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application [None req-665bb449-49c8-4c1c-87f7-9fdbf99c95f7 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 01 14:49:09.761296 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:49:09.761296 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:49:09.761296 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:49:09.761296 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:49:09.761296 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:49:09.761296 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:49:09.761296 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:49:09.761296 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:49:09.761296 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:49:09.761296 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:49:09.761296 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:49:09.761296 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 14:49:09.761296 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 14:49:09.761296 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 14:49:09.761296 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 14:49:09.762725 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:49:09.762725 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:49:09.762725 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 14:49:09.762725 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 14:49:09.762725 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 14:49:09.762725 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 14:49:09.762725 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:49:09.762725 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:49:09.762725 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:49:09.762725 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:49:09.762725 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:49:09.762725 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application with Lock( Sep 01 14:49:09.762725 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:49:09.762725 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:49:09.762725 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:49:09.762725 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:49:09.762725 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:49:09.762725 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:49:09.764459 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:49:09.764459 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:49:09.764459 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 14:49:09.764459 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 14:49:09.764459 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 14:49:09.764459 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 14:49:09.764459 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 14:49:09.764459 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 14:49:09.764459 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 01 14:49:09.764459 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Sep 01 14:49:09.764459 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 624/1239] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 1 14:49:09 2022] GET /identity/v3/users/placement => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:49:09.769222 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc329be6-2f31-45a6-afa9-a3c63d33fc4c None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:49:09.876667 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc329be6-2f31-45a6-afa9-a3c63d33fc4c None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:49:09.910997 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1b20af-6b9a-44d6-8587-130f8a7d34f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:49:09.911590 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1b20af-6b9a-44d6-8587-130f8a7d34f2 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:49:09.911874 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1b20af-6b9a-44d6-8587-130f8a7d34f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:49:09.912142 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1b20af-6b9a-44d6-8587-130f8a7d34f2 None admin] PATH_INFO: `/v3/users` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:49:09.912894 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1b20af-6b9a-44d6-8587-130f8a7d34f2 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81341) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:49:09.913207 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1b20af-6b9a-44d6-8587-130f8a7d34f2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:49:09.914615 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1b20af-6b9a-44d6-8587-130f8a7d34f2 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:49:09.933920 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 616/1240] 10.222.0.33 () {56 vars in 1143 bytes} [Thu Sep 1 14:49:09 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:49:09.939996 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665bb449-49c8-4c1c-87f7-9fdbf99c95f7 None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:49:10.066398 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665bb449-49c8-4c1c-87f7-9fdbf99c95f7 None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:49:10.109849 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7184e35-8972-45db-9e8c-b9eaafef7b24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:49:10.110518 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7184e35-8972-45db-9e8c-b9eaafef7b24 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:49:10.110807 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7184e35-8972-45db-9e8c-b9eaafef7b24 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:49:10.111176 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7184e35-8972-45db-9e8c-b9eaafef7b24 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:49:10.180838 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:49:10.188701 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7184e35-8972-45db-9e8c-b9eaafef7b24 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:49:10.190837 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7184e35-8972-45db-9e8c-b9eaafef7b24 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:49:10.305293 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 625/1241] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Sep 1 14:49:09 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 14:49:10.311165 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1b20af-6b9a-44d6-8587-130f8a7d34f2 None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:49:10.341013 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:49:10.424182 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1b20af-6b9a-44d6-8587-130f8a7d34f2 None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:49:10.461161 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e32e042-66ce-439c-91e5-7488f03f192b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:49:10.461777 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e32e042-66ce-439c-91e5-7488f03f192b None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:49:10.462050 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e32e042-66ce-439c-91e5-7488f03f192b None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:49:10.462340 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e32e042-66ce-439c-91e5-7488f03f192b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:49:10.473599 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e32e042-66ce-439c-91e5-7488f03f192b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:49:10.475276 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e32e042-66ce-439c-91e5-7488f03f192b None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:49:10.490157 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application [None req-6e32e042-66ce-439c-91e5-7488f03f192b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 14:49:10.490157 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:49:10.490157 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:49:10.490157 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:49:10.490157 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:49:10.490157 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:49:10.490157 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:49:10.490157 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:49:10.490157 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:49:10.490157 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:49:10.490157 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:49:10.490157 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:49:10.490157 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 14:49:10.490157 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 14:49:10.490157 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 14:49:10.490157 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 14:49:10.496003 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:49:10.496003 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:49:10.496003 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:49:10.496003 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:49:10.496003 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:49:10.496003 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:49:10.496003 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:49:10.496003 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application with Lock( Sep 01 14:49:10.496003 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:49:10.496003 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:49:10.496003 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:49:10.496003 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:49:10.496003 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:49:10.496003 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:49:10.496003 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:49:10.496003 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:49:10.496003 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 14:49:10.497130 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 14:49:10.497130 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 14:49:10.497130 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Sep 01 14:49:10.497130 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 617/1242] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 1 14:49:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 14:49:10.497409 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7184e35-8972-45db-9e8c-b9eaafef7b24 None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:49:10.602052 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:49:10.608710 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7184e35-8972-45db-9e8c-b9eaafef7b24 None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:49:10.643942 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e12a6fa-0721-4f64-be0f-b593b0a5da2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:49:10.644559 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e12a6fa-0721-4f64-be0f-b593b0a5da2f None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:49:10.644836 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e12a6fa-0721-4f64-be0f-b593b0a5da2f None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:49:10.645125 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e12a6fa-0721-4f64-be0f-b593b0a5da2f None admin] PATH_INFO: `/v3/domains` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:49:10.645741 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e12a6fa-0721-4f64-be0f-b593b0a5da2f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:49:10.646033 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e12a6fa-0721-4f64-be0f-b593b0a5da2f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:49:10.647453 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e12a6fa-0721-4f64-be0f-b593b0a5da2f None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:49:10.660105 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 626/1243] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Sep 1 14:49:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 14:49:10.665495 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e32e042-66ce-439c-91e5-7488f03f192b None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:49:10.748103 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:49:10.774682 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e32e042-66ce-439c-91e5-7488f03f192b None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:49:10.813150 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83af2382-cc23-4f4a-b5e2-b8c701d10df8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:49:10.813856 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83af2382-cc23-4f4a-b5e2-b8c701d10df8 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:49:10.814063 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83af2382-cc23-4f4a-b5e2-b8c701d10df8 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:49:10.814353 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83af2382-cc23-4f4a-b5e2-b8c701d10df8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:49:10.822681 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83af2382-cc23-4f4a-b5e2-b8c701d10df8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:49:10.824157 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83af2382-cc23-4f4a-b5e2-b8c701d10df8 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:49:10.837201 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application [None req-83af2382-cc23-4f4a-b5e2-b8c701d10df8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 14:49:10.837201 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:49:10.837201 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:49:10.837201 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:49:10.837201 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:49:10.837201 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:49:10.837201 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:49:10.837201 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:49:10.837201 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:49:10.837201 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:49:10.837201 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:49:10.837201 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:49:10.837201 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 14:49:10.837201 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 14:49:10.837201 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 14:49:10.838489 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 14:49:10.838489 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:49:10.838489 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:49:10.838489 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:49:10.838489 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:49:10.838489 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:49:10.838489 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:49:10.838489 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:49:10.838489 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application with Lock( Sep 01 14:49:10.838489 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:49:10.838489 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:49:10.838489 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:49:10.838489 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:49:10.838489 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:49:10.838489 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:49:10.838489 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:49:10.838489 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:49:10.839540 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 14:49:10.839540 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 14:49:10.839540 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 14:49:10.839540 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 14:49:10.839540 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 14:49:10.839540 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 14:49:10.839540 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 14:49:10.839540 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Sep 01 14:49:10.839540 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 618/1244] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Sep 1 14:49:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:49:10.845050 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e12a6fa-0721-4f64-be0f-b593b0a5da2f None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:49:10.957366 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e12a6fa-0721-4f64-be0f-b593b0a5da2f None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:49:10.993218 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5b0dd0-1e26-4e20-8f0c-6a4d65e7e5aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:49:10.993822 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5b0dd0-1e26-4e20-8f0c-6a4d65e7e5aa None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:49:10.994098 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5b0dd0-1e26-4e20-8f0c-6a4d65e7e5aa None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:49:10.994405 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5b0dd0-1e26-4e20-8f0c-6a4d65e7e5aa None admin] PATH_INFO: `/v3/projects` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:49:10.995124 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5b0dd0-1e26-4e20-8f0c-6a4d65e7e5aa None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:49:10.995399 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5b0dd0-1e26-4e20-8f0c-6a4d65e7e5aa None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:49:10.996905 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5b0dd0-1e26-4e20-8f0c-6a4d65e7e5aa None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:49:11.010178 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 627/1245] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Sep 1 14:49:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:49:11.016992 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83af2382-cc23-4f4a-b5e2-b8c701d10df8 None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:49:11.140768 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83af2382-cc23-4f4a-b5e2-b8c701d10df8 None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:49:11.177282 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c165a6ba-e7d3-4068-8ea3-d05393a183f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:49:11.177954 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c165a6ba-e7d3-4068-8ea3-d05393a183f7 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:49:11.178250 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c165a6ba-e7d3-4068-8ea3-d05393a183f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:49:11.178541 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c165a6ba-e7d3-4068-8ea3-d05393a183f7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:49:11.179417 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c165a6ba-e7d3-4068-8ea3-d05393a183f7 None admin] RBAC: Adding query filter params (role.id=5e3f992e804046f381b3de8bf2a512e2, scope.project.id=e8c85b4bd17e4e89b56bf0f831b474d5, user.id=7f312c8dcabe48ef9ea453364ade38ff) {{(pid=81341) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:49:11.179737 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c165a6ba-e7d3-4068-8ea3-d05393a183f7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:49:11.181432 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c165a6ba-e7d3-4068-8ea3-d05393a183f7 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:49:11.187702 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 619/1246] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Sep 1 14:49:11 2022] GET /identity/v3/role_assignments?user.id=7f312c8dcabe48ef9ea453364ade38ff&scope.project.id=e8c85b4bd17e4e89b56bf0f831b474d5&role.id=5e3f992e804046f381b3de8bf2a512e2 => generated 608 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:49:13.591404 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af127de2-df30-4eff-a128-4d5fcb3246ad None None] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:49:13.591935 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af127de2-df30-4eff-a128-4d5fcb3246ad None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:49:13.591935 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af127de2-df30-4eff-a128-4d5fcb3246ad None None] PATH_INFO: `/` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:49:13.593368 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 628/1247] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 1 14:49:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:49:13.600995 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086efabc-f8e0-4257-927c-c4039419a75e None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:49:13.601474 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086efabc-f8e0-4257-927c-c4039419a75e None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:49:13.601742 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086efabc-f8e0-4257-927c-c4039419a75e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:49:13.608811 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:49:13.622264 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-086efabc-f8e0-4257-927c-c4039419a75e None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:49:13.623500 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-086efabc-f8e0-4257-927c-c4039419a75e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:49:13.701962 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 620/1248] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 1 14:49:13 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 14:49:13.966822 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af127de2-df30-4eff-a128-4d5fcb3246ad None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:49:13.990362 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:49:14.097247 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af127de2-df30-4eff-a128-4d5fcb3246ad None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:49:14.139514 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16904e1-a8b5-454e-b9e3-5f6b5611e1cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:49:14.140185 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16904e1-a8b5-454e-b9e3-5f6b5611e1cf None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:49:14.140465 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16904e1-a8b5-454e-b9e3-5f6b5611e1cf None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:49:14.140742 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16904e1-a8b5-454e-b9e3-5f6b5611e1cf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:49:14.152049 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16904e1-a8b5-454e-b9e3-5f6b5611e1cf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 14:49:14.157416 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application [None req-a16904e1-a8b5-454e-b9e3-5f6b5611e1cf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 14:49:14.157416 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:49:14.157416 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:49:14.157416 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:49:14.157416 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:49:14.157416 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:49:14.157416 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:49:14.157416 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:49:14.157416 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:49:14.157416 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:49:14.157416 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:49:14.157416 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:49:14.157416 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 14:49:14.157416 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 14:49:14.157416 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 14:49:14.157416 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 14:49:14.158523 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 14:49:14.158523 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 14:49:14.158523 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 14:49:14.158523 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 14:49:14.158523 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:49:14.158523 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:49:14.158523 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:49:14.158523 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:49:14.158523 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:49:14.158523 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:49:14.158523 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:49:14.158523 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application with Lock( Sep 01 14:49:14.158523 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:49:14.158523 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:49:14.158523 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:49:14.158523 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:49:14.159873 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:49:14.159873 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:49:14.159873 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:49:14.159873 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:49:14.159873 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 14:49:14.159873 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 14:49:14.159873 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 14:49:14.159873 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 14:49:14.159873 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 14:49:14.159873 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 14:49:14.159873 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 14:49:14.159873 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Sep 01 14:49:14.159873 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 629/1249] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Sep 1 14:49:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:49:14.164786 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086efabc-f8e0-4257-927c-c4039419a75e None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:49:14.274035 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086efabc-f8e0-4257-927c-c4039419a75e None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:49:14.304350 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:49:14.310816 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d831c4a9-1745-40c0-b3c4-50c51eb3cf55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:49:14.311479 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d831c4a9-1745-40c0-b3c4-50c51eb3cf55 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:49:14.311763 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d831c4a9-1745-40c0-b3c4-50c51eb3cf55 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:49:14.312038 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d831c4a9-1745-40c0-b3c4-50c51eb3cf55 None admin] PATH_INFO: `/v3/roles` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:49:14.312726 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d831c4a9-1745-40c0-b3c4-50c51eb3cf55 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81341) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:49:14.313062 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d831c4a9-1745-40c0-b3c4-50c51eb3cf55 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:49:14.314622 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d831c4a9-1745-40c0-b3c4-50c51eb3cf55 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:49:14.324087 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 621/1250] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Sep 1 14:49:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:49:14.329885 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16904e1-a8b5-454e-b9e3-5f6b5611e1cf None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:49:14.413720 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:49:14.442627 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16904e1-a8b5-454e-b9e3-5f6b5611e1cf None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:49:14.480805 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542d2a7e-cbf8-427a-97b7-c7f6ee7379b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:49:14.481446 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542d2a7e-cbf8-427a-97b7-c7f6ee7379b5 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:49:14.481711 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542d2a7e-cbf8-427a-97b7-c7f6ee7379b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:49:14.481976 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542d2a7e-cbf8-427a-97b7-c7f6ee7379b5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:49:14.494882 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542d2a7e-cbf8-427a-97b7-c7f6ee7379b5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:49:14.496807 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542d2a7e-cbf8-427a-97b7-c7f6ee7379b5 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:49:14.513777 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application [None req-542d2a7e-cbf8-427a-97b7-c7f6ee7379b5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 14:49:14.513777 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:49:14.513777 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:49:14.513777 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:49:14.513777 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:49:14.513777 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:49:14.513777 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:49:14.513777 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:49:14.513777 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:49:14.513777 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:49:14.513777 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:49:14.513777 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:49:14.513777 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 14:49:14.513777 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 14:49:14.513777 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 14:49:14.513777 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 14:49:14.515337 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:49:14.515337 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:49:14.515337 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:49:14.515337 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:49:14.515337 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:49:14.515337 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:49:14.515337 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:49:14.515337 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application with Lock( Sep 01 14:49:14.515337 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:49:14.515337 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:49:14.515337 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:49:14.515337 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:49:14.515337 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:49:14.515337 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:49:14.515337 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:49:14.515337 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:49:14.515337 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 14:49:14.516419 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 14:49:14.516419 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 14:49:14.516419 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Sep 01 14:49:14.516419 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 630/1251] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 1 14:49:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:49:14.520768 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d831c4a9-1745-40c0-b3c4-50c51eb3cf55 None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:49:14.635638 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d831c4a9-1745-40c0-b3c4-50c51eb3cf55 None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:49:14.677136 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c59705f-fdaa-4a2c-a526-6f82e7305c91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:49:14.677753 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c59705f-fdaa-4a2c-a526-6f82e7305c91 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:49:14.678049 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c59705f-fdaa-4a2c-a526-6f82e7305c91 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:49:14.678385 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c59705f-fdaa-4a2c-a526-6f82e7305c91 None admin] PATH_INFO: `/v3/domains` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:49:14.679153 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c59705f-fdaa-4a2c-a526-6f82e7305c91 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81341) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:49:14.679473 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c59705f-fdaa-4a2c-a526-6f82e7305c91 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:49:14.681074 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c59705f-fdaa-4a2c-a526-6f82e7305c91 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:49:14.693650 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 622/1252] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Sep 1 14:49:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 14:49:14.699767 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542d2a7e-cbf8-427a-97b7-c7f6ee7379b5 None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:49:14.810774 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542d2a7e-cbf8-427a-97b7-c7f6ee7379b5 None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:49:14.839838 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:49:14.849268 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea35d315-8d50-478c-8239-aeefc014fb7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:49:14.849895 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea35d315-8d50-478c-8239-aeefc014fb7f None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:49:14.850175 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea35d315-8d50-478c-8239-aeefc014fb7f None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:49:14.850473 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea35d315-8d50-478c-8239-aeefc014fb7f None admin] PATH_INFO: `/v3/users/placement` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:49:14.857948 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea35d315-8d50-478c-8239-aeefc014fb7f None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:49:14.859423 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea35d315-8d50-478c-8239-aeefc014fb7f None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:49:14.871786 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application [None req-ea35d315-8d50-478c-8239-aeefc014fb7f None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 01 14:49:14.871786 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:49:14.871786 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:49:14.871786 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:49:14.871786 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:49:14.871786 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:49:14.871786 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:49:14.871786 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:49:14.871786 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:49:14.871786 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:49:14.871786 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:49:14.871786 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:49:14.871786 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 14:49:14.871786 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 14:49:14.871786 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 14:49:14.871786 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 14:49:14.872898 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:49:14.872898 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:49:14.872898 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 14:49:14.872898 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 14:49:14.872898 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 14:49:14.872898 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 14:49:14.872898 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:49:14.872898 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:49:14.872898 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:49:14.872898 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:49:14.872898 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:49:14.872898 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application with Lock( Sep 01 14:49:14.872898 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:49:14.872898 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:49:14.872898 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:49:14.872898 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:49:14.872898 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:49:14.872898 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:49:14.873915 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:49:14.873915 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:49:14.873915 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 14:49:14.873915 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 14:49:14.873915 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 14:49:14.873915 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 14:49:14.873915 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 14:49:14.873915 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 14:49:14.873915 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 01 14:49:14.873915 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Sep 01 14:49:14.873915 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 631/1253] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 1 14:49:14 2022] GET /identity/v3/users/placement => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:49:14.879236 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c59705f-fdaa-4a2c-a526-6f82e7305c91 None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:49:14.937559 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:49:14.987975 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c59705f-fdaa-4a2c-a526-6f82e7305c91 None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:49:15.025387 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51ca0e8-a535-4539-8995-2054d64eed28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:49:15.025994 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51ca0e8-a535-4539-8995-2054d64eed28 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:49:15.026296 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51ca0e8-a535-4539-8995-2054d64eed28 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:49:15.026586 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51ca0e8-a535-4539-8995-2054d64eed28 None admin] PATH_INFO: `/v3/users` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:49:15.027297 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51ca0e8-a535-4539-8995-2054d64eed28 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81341) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:49:15.027595 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51ca0e8-a535-4539-8995-2054d64eed28 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:49:15.029033 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51ca0e8-a535-4539-8995-2054d64eed28 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:49:15.050966 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 623/1254] 10.222.0.33 () {56 vars in 1143 bytes} [Thu Sep 1 14:49:14 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:49:15.057004 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea35d315-8d50-478c-8239-aeefc014fb7f None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:49:15.185510 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea35d315-8d50-478c-8239-aeefc014fb7f None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:49:15.226330 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe19915-4725-472b-b898-297422189b53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:49:15.226895 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe19915-4725-472b-b898-297422189b53 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:49:15.227198 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe19915-4725-472b-b898-297422189b53 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:49:15.227563 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe19915-4725-472b-b898-297422189b53 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:49:15.296037 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:49:15.314267 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe19915-4725-472b-b898-297422189b53 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:49:15.314267 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe19915-4725-472b-b898-297422189b53 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:49:15.417561 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 632/1255] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Sep 1 14:49:15 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 01 14:49:15.423665 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51ca0e8-a535-4539-8995-2054d64eed28 None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:49:15.529448 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51ca0e8-a535-4539-8995-2054d64eed28 None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:49:15.543154 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:49:15.565647 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8af343-d65b-4c58-a115-0478adce82f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:49:15.566320 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8af343-d65b-4c58-a115-0478adce82f1 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:49:15.566603 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8af343-d65b-4c58-a115-0478adce82f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:49:15.566876 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8af343-d65b-4c58-a115-0478adce82f1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:49:15.578289 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8af343-d65b-4c58-a115-0478adce82f1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:49:15.579780 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8af343-d65b-4c58-a115-0478adce82f1 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:49:15.595560 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application [None req-6c8af343-d65b-4c58-a115-0478adce82f1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 14:49:15.595560 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:49:15.595560 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:49:15.595560 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:49:15.595560 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:49:15.595560 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:49:15.595560 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:49:15.595560 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:49:15.595560 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:49:15.595560 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:49:15.595560 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:49:15.595560 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:49:15.595560 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 14:49:15.595560 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 14:49:15.595560 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 14:49:15.595560 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 14:49:15.596690 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:49:15.596690 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:49:15.596690 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:49:15.596690 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:49:15.596690 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:49:15.596690 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:49:15.596690 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:49:15.596690 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application with Lock( Sep 01 14:49:15.596690 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:49:15.596690 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:49:15.596690 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:49:15.596690 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:49:15.596690 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:49:15.596690 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:49:15.596690 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:49:15.596690 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:49:15.596690 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 14:49:15.598055 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 14:49:15.598055 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 14:49:15.598055 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Sep 01 14:49:15.598055 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 624/1256] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 1 14:49:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 14:49:15.602586 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe19915-4725-472b-b898-297422189b53 None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:49:15.694556 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:49:15.706861 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe19915-4725-472b-b898-297422189b53 None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:49:15.741190 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca575ddb-6d57-4137-b290-8151bed95cd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:49:15.741785 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca575ddb-6d57-4137-b290-8151bed95cd2 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:49:15.742062 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca575ddb-6d57-4137-b290-8151bed95cd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:49:15.742353 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca575ddb-6d57-4137-b290-8151bed95cd2 None admin] PATH_INFO: `/v3/domains` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:49:15.742990 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca575ddb-6d57-4137-b290-8151bed95cd2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:49:15.743288 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca575ddb-6d57-4137-b290-8151bed95cd2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:49:15.744646 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca575ddb-6d57-4137-b290-8151bed95cd2 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:49:15.756488 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 633/1257] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Sep 1 14:49:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:49:15.762103 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8af343-d65b-4c58-a115-0478adce82f1 None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:49:15.865916 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8af343-d65b-4c58-a115-0478adce82f1 None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:49:15.899293 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e47fd7-4aff-4162-a85f-69ca3b2e2dbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:49:15.899920 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e47fd7-4aff-4162-a85f-69ca3b2e2dbb None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:49:15.900190 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e47fd7-4aff-4162-a85f-69ca3b2e2dbb None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:49:15.900455 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e47fd7-4aff-4162-a85f-69ca3b2e2dbb None admin] PATH_INFO: `/v3/projects/service` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:49:15.907020 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e47fd7-4aff-4162-a85f-69ca3b2e2dbb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:49:15.908462 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e47fd7-4aff-4162-a85f-69ca3b2e2dbb None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:49:15.919464 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application [None req-d4e47fd7-4aff-4162-a85f-69ca3b2e2dbb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 14:49:15.919464 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:49:15.919464 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:49:15.919464 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:49:15.919464 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:49:15.919464 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:49:15.919464 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:49:15.919464 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:49:15.919464 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:49:15.919464 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:49:15.919464 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:49:15.919464 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:49:15.919464 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 14:49:15.919464 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 14:49:15.919464 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 14:49:15.920556 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 14:49:15.920556 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:49:15.920556 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:49:15.920556 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:49:15.920556 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:49:15.920556 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:49:15.920556 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:49:15.920556 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:49:15.920556 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application with Lock( Sep 01 14:49:15.920556 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:49:15.920556 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:49:15.920556 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:49:15.920556 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:49:15.920556 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:49:15.920556 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:49:15.920556 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:49:15.920556 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:49:15.921510 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 14:49:15.921510 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 14:49:15.921510 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 14:49:15.921510 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 14:49:15.921510 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 14:49:15.921510 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 14:49:15.921510 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 14:49:15.921510 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Sep 01 14:49:15.921510 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 625/1258] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Sep 1 14:49:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:49:15.927112 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca575ddb-6d57-4137-b290-8151bed95cd2 None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:49:16.024856 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca575ddb-6d57-4137-b290-8151bed95cd2 None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:49:16.062627 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0d3c79-ea5e-45dc-a178-6cd01e856ca4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:49:16.063231 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0d3c79-ea5e-45dc-a178-6cd01e856ca4 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:49:16.063625 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0d3c79-ea5e-45dc-a178-6cd01e856ca4 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:49:16.064079 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0d3c79-ea5e-45dc-a178-6cd01e856ca4 None admin] PATH_INFO: `/v3/projects` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:49:16.064607 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0d3c79-ea5e-45dc-a178-6cd01e856ca4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:49:16.064944 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0d3c79-ea5e-45dc-a178-6cd01e856ca4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:49:16.066449 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0d3c79-ea5e-45dc-a178-6cd01e856ca4 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:49:16.080615 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 634/1259] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Sep 1 14:49:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:49:16.088337 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e47fd7-4aff-4162-a85f-69ca3b2e2dbb None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:49:16.114208 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:49:16.186502 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e47fd7-4aff-4162-a85f-69ca3b2e2dbb None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:49:16.218023 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cd559e-ba86-4eb3-a918-7893b11e8976 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:49:16.218633 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cd559e-ba86-4eb3-a918-7893b11e8976 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:49:16.218911 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cd559e-ba86-4eb3-a918-7893b11e8976 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:49:16.219180 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cd559e-ba86-4eb3-a918-7893b11e8976 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:49:16.219888 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0cd559e-ba86-4eb3-a918-7893b11e8976 None admin] RBAC: Adding query filter params (role.id=2a2adb769e55476b82e5014897e0a9c0, scope.project.id=e8c85b4bd17e4e89b56bf0f831b474d5, user.id=7f312c8dcabe48ef9ea453364ade38ff) {{(pid=81341) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:49:16.220195 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0cd559e-ba86-4eb3-a918-7893b11e8976 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:49:16.222192 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0cd559e-ba86-4eb3-a918-7893b11e8976 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:49:16.227391 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 626/1260] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Sep 1 14:49:16 2022] GET /identity/v3/role_assignments?user.id=7f312c8dcabe48ef9ea453364ade38ff&scope.project.id=e8c85b4bd17e4e89b56bf0f831b474d5&role.id=2a2adb769e55476b82e5014897e0a9c0 => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:49:18.063165 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d29a181-99d4-4e8a-a831-266d4e20e0e7 None None] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:49:18.063707 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d29a181-99d4-4e8a-a831-266d4e20e0e7 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:49:18.063707 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d29a181-99d4-4e8a-a831-266d4e20e0e7 None None] PATH_INFO: `/` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:49:18.065067 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 635/1261] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 1 14:49:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:49:18.071668 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97956784-a1c3-438a-8fd6-3e4d6c5dc000 None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:49:18.072137 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97956784-a1c3-438a-8fd6-3e4d6c5dc000 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:49:18.072380 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97956784-a1c3-438a-8fd6-3e4d6c5dc000 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:49:18.092900 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-97956784-a1c3-438a-8fd6-3e4d6c5dc000 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:49:18.094080 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-97956784-a1c3-438a-8fd6-3e4d6c5dc000 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:49:18.158710 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 627/1262] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 1 14:49:18 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 14:49:18.420004 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d29a181-99d4-4e8a-a831-266d4e20e0e7 None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:49:18.431967 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:49:18.525031 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d29a181-99d4-4e8a-a831-266d4e20e0e7 None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:49:18.557144 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a475315-f094-4b74-8107-f162008da531 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:49:18.557750 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a475315-f094-4b74-8107-f162008da531 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:49:18.558026 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a475315-f094-4b74-8107-f162008da531 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:49:18.558319 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a475315-f094-4b74-8107-f162008da531 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:49:18.568839 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a475315-f094-4b74-8107-f162008da531 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 14:49:18.573562 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application [None req-2a475315-f094-4b74-8107-f162008da531 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 14:49:18.573562 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:49:18.573562 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:49:18.573562 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:49:18.573562 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:49:18.573562 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:49:18.573562 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:49:18.573562 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:49:18.573562 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:49:18.573562 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:49:18.573562 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:49:18.573562 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:49:18.573562 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 14:49:18.573562 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 14:49:18.573562 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 14:49:18.573562 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 14:49:18.574674 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 14:49:18.574674 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 14:49:18.574674 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 14:49:18.574674 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 14:49:18.574674 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:49:18.574674 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:49:18.574674 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:49:18.574674 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:49:18.574674 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:49:18.574674 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:49:18.574674 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:49:18.574674 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application with Lock( Sep 01 14:49:18.574674 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:49:18.574674 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:49:18.574674 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:49:18.574674 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:49:18.575729 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:49:18.575729 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:49:18.575729 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:49:18.575729 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:49:18.575729 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 14:49:18.575729 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 14:49:18.575729 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 14:49:18.575729 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 14:49:18.575729 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 14:49:18.575729 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 14:49:18.575729 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 14:49:18.575729 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Sep 01 14:49:18.575729 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 636/1263] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Sep 1 14:49:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:49:18.581318 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97956784-a1c3-438a-8fd6-3e4d6c5dc000 None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:49:18.599056 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:49:18.677744 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97956784-a1c3-438a-8fd6-3e4d6c5dc000 None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:49:18.710751 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656d09b9-2342-41b3-89f2-57fb3d3995be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:49:18.711347 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656d09b9-2342-41b3-89f2-57fb3d3995be None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:49:18.711626 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656d09b9-2342-41b3-89f2-57fb3d3995be None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:49:18.711896 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656d09b9-2342-41b3-89f2-57fb3d3995be None admin] PATH_INFO: `/v3/roles` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:49:18.712555 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656d09b9-2342-41b3-89f2-57fb3d3995be None admin] RBAC: Adding query filter params (name=admin) {{(pid=81341) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:49:18.712857 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656d09b9-2342-41b3-89f2-57fb3d3995be None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:49:18.714262 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656d09b9-2342-41b3-89f2-57fb3d3995be None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:49:18.722300 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 628/1264] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Sep 1 14:49:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:49:18.727891 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a475315-f094-4b74-8107-f162008da531 None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:49:18.792219 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:49:18.824822 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a475315-f094-4b74-8107-f162008da531 None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:49:18.856259 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567d5651-1693-4099-a59c-ee08ba74aeca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:49:18.856856 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567d5651-1693-4099-a59c-ee08ba74aeca None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:49:18.857128 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567d5651-1693-4099-a59c-ee08ba74aeca None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:49:18.857392 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567d5651-1693-4099-a59c-ee08ba74aeca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:49:18.867388 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567d5651-1693-4099-a59c-ee08ba74aeca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:49:18.868819 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567d5651-1693-4099-a59c-ee08ba74aeca None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:49:18.883460 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application [None req-567d5651-1693-4099-a59c-ee08ba74aeca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 14:49:18.883460 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:49:18.883460 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:49:18.883460 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:49:18.883460 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:49:18.883460 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:49:18.883460 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:49:18.883460 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:49:18.883460 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:49:18.883460 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:49:18.883460 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:49:18.883460 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:49:18.883460 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 14:49:18.883460 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 14:49:18.883460 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 14:49:18.883460 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 14:49:18.884564 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:49:18.884564 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:49:18.884564 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:49:18.884564 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:49:18.884564 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:49:18.884564 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:49:18.884564 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:49:18.884564 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application with Lock( Sep 01 14:49:18.884564 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:49:18.884564 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:49:18.884564 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:49:18.884564 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:49:18.884564 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:49:18.884564 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:49:18.884564 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:49:18.884564 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:49:18.884564 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 14:49:18.885490 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 14:49:18.885490 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 14:49:18.885490 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Sep 01 14:49:18.885490 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 637/1265] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 1 14:49:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:49:18.890120 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656d09b9-2342-41b3-89f2-57fb3d3995be None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:49:18.966493 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:49:18.985538 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656d09b9-2342-41b3-89f2-57fb3d3995be None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:49:19.018402 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae65685-cd17-4d5c-90da-0f3d9bbab418 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:49:19.018996 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae65685-cd17-4d5c-90da-0f3d9bbab418 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:49:19.019277 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae65685-cd17-4d5c-90da-0f3d9bbab418 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:49:19.019547 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae65685-cd17-4d5c-90da-0f3d9bbab418 None admin] PATH_INFO: `/v3/domains` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:49:19.020175 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae65685-cd17-4d5c-90da-0f3d9bbab418 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81341) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:49:19.020474 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae65685-cd17-4d5c-90da-0f3d9bbab418 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:49:19.021870 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae65685-cd17-4d5c-90da-0f3d9bbab418 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:49:19.033089 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 629/1266] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Sep 1 14:49:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:49:19.039432 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567d5651-1693-4099-a59c-ee08ba74aeca None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:49:19.147339 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567d5651-1693-4099-a59c-ee08ba74aeca None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:49:19.167059 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:49:19.181164 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2846ba-7f0b-44a3-a660-c22c5cc55d97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:49:19.181776 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2846ba-7f0b-44a3-a660-c22c5cc55d97 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:49:19.182048 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2846ba-7f0b-44a3-a660-c22c5cc55d97 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:49:19.182346 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2846ba-7f0b-44a3-a660-c22c5cc55d97 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:49:19.188948 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2846ba-7f0b-44a3-a660-c22c5cc55d97 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:49:19.190436 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2846ba-7f0b-44a3-a660-c22c5cc55d97 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:49:19.201680 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application [None req-9e2846ba-7f0b-44a3-a660-c22c5cc55d97 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 01 14:49:19.201680 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:49:19.201680 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:49:19.201680 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:49:19.201680 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:49:19.201680 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:49:19.201680 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:49:19.201680 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:49:19.201680 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:49:19.201680 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:49:19.201680 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:49:19.201680 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:49:19.201680 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 14:49:19.201680 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 14:49:19.201680 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 14:49:19.201680 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 14:49:19.202810 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:49:19.202810 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:49:19.202810 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 14:49:19.202810 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 14:49:19.202810 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 14:49:19.202810 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 14:49:19.202810 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:49:19.202810 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:49:19.202810 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:49:19.202810 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:49:19.202810 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:49:19.202810 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application with Lock( Sep 01 14:49:19.202810 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:49:19.202810 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:49:19.202810 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:49:19.202810 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:49:19.202810 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:49:19.202810 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:49:19.204003 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:49:19.204003 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:49:19.204003 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 14:49:19.204003 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 14:49:19.204003 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 14:49:19.204003 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 14:49:19.204003 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 14:49:19.204003 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 14:49:19.204003 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 01 14:49:19.204003 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Sep 01 14:49:19.204003 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 638/1267] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 1 14:49:19 2022] GET /identity/v3/users/placement => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 14:49:19.208767 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae65685-cd17-4d5c-90da-0f3d9bbab418 None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:49:19.303722 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae65685-cd17-4d5c-90da-0f3d9bbab418 None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:49:19.336305 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4559c40-e57c-4d5b-93da-5f955bbe5a95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:49:19.336881 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4559c40-e57c-4d5b-93da-5f955bbe5a95 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:49:19.337152 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4559c40-e57c-4d5b-93da-5f955bbe5a95 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:49:19.337414 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4559c40-e57c-4d5b-93da-5f955bbe5a95 None admin] PATH_INFO: `/v3/users` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:49:19.338115 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4559c40-e57c-4d5b-93da-5f955bbe5a95 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81341) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:49:19.338437 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4559c40-e57c-4d5b-93da-5f955bbe5a95 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:49:19.339853 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4559c40-e57c-4d5b-93da-5f955bbe5a95 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:49:19.348571 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:49:19.356771 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 630/1268] 10.222.0.33 () {56 vars in 1143 bytes} [Thu Sep 1 14:49:19 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:49:19.363015 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2846ba-7f0b-44a3-a660-c22c5cc55d97 None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:49:19.460119 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2846ba-7f0b-44a3-a660-c22c5cc55d97 None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:49:19.492043 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0440ac9-33fe-4b66-8975-e50437897ff9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:49:19.492658 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0440ac9-33fe-4b66-8975-e50437897ff9 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:49:19.492936 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0440ac9-33fe-4b66-8975-e50437897ff9 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:49:19.493202 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0440ac9-33fe-4b66-8975-e50437897ff9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:49:19.503395 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0440ac9-33fe-4b66-8975-e50437897ff9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:49:19.504842 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0440ac9-33fe-4b66-8975-e50437897ff9 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:49:19.519105 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application [None req-c0440ac9-33fe-4b66-8975-e50437897ff9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 14:49:19.519105 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:49:19.519105 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:49:19.519105 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:49:19.519105 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:49:19.519105 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:49:19.519105 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:49:19.519105 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:49:19.519105 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:49:19.519105 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:49:19.519105 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:49:19.519105 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:49:19.519105 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 14:49:19.519105 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 14:49:19.519105 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 14:49:19.519105 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 14:49:19.520273 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:49:19.520273 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:49:19.520273 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:49:19.520273 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:49:19.520273 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:49:19.520273 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:49:19.520273 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:49:19.520273 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application with Lock( Sep 01 14:49:19.520273 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:49:19.520273 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:49:19.520273 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:49:19.520273 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:49:19.520273 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:49:19.520273 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:49:19.520273 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:49:19.520273 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:49:19.520273 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 14:49:19.521537 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 14:49:19.521537 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 14:49:19.521537 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Sep 01 14:49:19.521537 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 639/1269] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 1 14:49:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:49:19.526352 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4559c40-e57c-4d5b-93da-5f955bbe5a95 None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:49:19.622168 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4559c40-e57c-4d5b-93da-5f955bbe5a95 None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:49:19.654596 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5854268-19ae-4066-92b1-0acea742a465 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:49:19.655191 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5854268-19ae-4066-92b1-0acea742a465 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:49:19.655464 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5854268-19ae-4066-92b1-0acea742a465 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:49:19.655731 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5854268-19ae-4066-92b1-0acea742a465 None admin] PATH_INFO: `/v3/domains` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:49:19.656361 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5854268-19ae-4066-92b1-0acea742a465 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81341) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:49:19.656656 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5854268-19ae-4066-92b1-0acea742a465 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:49:19.658033 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5854268-19ae-4066-92b1-0acea742a465 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:49:19.669893 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 631/1270] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Sep 1 14:49:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:49:19.675613 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0440ac9-33fe-4b66-8975-e50437897ff9 None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:49:19.707067 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:49:19.774536 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0440ac9-33fe-4b66-8975-e50437897ff9 None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:49:19.806682 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a0f632-bbc1-4518-b7fe-719f5ec6cbdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:49:19.807292 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a0f632-bbc1-4518-b7fe-719f5ec6cbdf None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:49:19.807565 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a0f632-bbc1-4518-b7fe-719f5ec6cbdf None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:49:19.807830 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a0f632-bbc1-4518-b7fe-719f5ec6cbdf None admin] PATH_INFO: `/v3/projects/service` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:49:19.814221 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a0f632-bbc1-4518-b7fe-719f5ec6cbdf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:49:19.816154 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a0f632-bbc1-4518-b7fe-719f5ec6cbdf None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:49:19.827206 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application [None req-64a0f632-bbc1-4518-b7fe-719f5ec6cbdf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 14:49:19.827206 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:49:19.827206 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:49:19.827206 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:49:19.827206 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:49:19.827206 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:49:19.827206 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:49:19.827206 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:49:19.827206 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:49:19.827206 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:49:19.827206 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:49:19.827206 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:49:19.827206 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 14:49:19.827206 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 14:49:19.827206 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 14:49:19.828124 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 14:49:19.828124 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:49:19.828124 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:49:19.828124 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:49:19.828124 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:49:19.828124 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:49:19.828124 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:49:19.828124 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:49:19.828124 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application with Lock( Sep 01 14:49:19.828124 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:49:19.828124 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:49:19.828124 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:49:19.828124 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:49:19.828124 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:49:19.828124 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:49:19.828124 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:49:19.828124 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:49:19.829134 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 14:49:19.829134 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 14:49:19.829134 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 14:49:19.829134 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 14:49:19.829134 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 14:49:19.829134 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 14:49:19.829134 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 14:49:19.829134 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Sep 01 14:49:19.829134 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 640/1271] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Sep 1 14:49:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:49:19.834787 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5854268-19ae-4066-92b1-0acea742a465 None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:49:19.891624 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:49:19.932492 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5854268-19ae-4066-92b1-0acea742a465 None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:49:19.965497 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60ce3a3-4ccf-49b4-bc54-16e0f2013c8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:49:19.966084 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60ce3a3-4ccf-49b4-bc54-16e0f2013c8b None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:49:19.966387 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60ce3a3-4ccf-49b4-bc54-16e0f2013c8b None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:49:19.966656 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60ce3a3-4ccf-49b4-bc54-16e0f2013c8b None admin] PATH_INFO: `/v3/projects` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:49:19.967335 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60ce3a3-4ccf-49b4-bc54-16e0f2013c8b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81341) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:49:19.967633 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60ce3a3-4ccf-49b4-bc54-16e0f2013c8b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:49:19.969051 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60ce3a3-4ccf-49b4-bc54-16e0f2013c8b None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:49:19.980500 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 632/1272] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Sep 1 14:49:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:49:19.986805 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a0f632-bbc1-4518-b7fe-719f5ec6cbdf None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:49:20.084415 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:49:20.093224 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a0f632-bbc1-4518-b7fe-719f5ec6cbdf None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:49:20.126478 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f1f6f5-90a5-4b39-b512-2becceccc7e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:49:20.127121 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f1f6f5-90a5-4b39-b512-2becceccc7e3 None admin] REQUEST_METHOD: `PUT` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:49:20.127458 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f1f6f5-90a5-4b39-b512-2becceccc7e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:49:20.128237 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f1f6f5-90a5-4b39-b512-2becceccc7e3 None admin] PATH_INFO: `/v3/projects/e8c85b4bd17e4e89b56bf0f831b474d5/users/7f312c8dcabe48ef9ea453364ade38ff/roles/2a2adb769e55476b82e5014897e0a9c0` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:49:20.130105 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f1f6f5-90a5-4b39-b512-2becceccc7e3 None admin] RBAC: Authorizing `identity:create_grant(project_id=e8c85b4bd17e4e89b56bf0f831b474d5, user_id=7f312c8dcabe48ef9ea453364ade38ff, role_id=2a2adb769e55476b82e5014897e0a9c0)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:49:20.131554 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f1f6f5-90a5-4b39-b512-2becceccc7e3 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:49:20.144044 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 641/1273] 10.222.0.33 () {58 vars in 1439 bytes} [Thu Sep 1 14:49:19 2022] PUT /identity/v3/projects/e8c85b4bd17e4e89b56bf0f831b474d5/users/7f312c8dcabe48ef9ea453364ade38ff/roles/2a2adb769e55476b82e5014897e0a9c0 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:49:21.998802 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004202ea-2290-43d2-ad97-3e2759876269 None None] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:49:21.999300 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004202ea-2290-43d2-ad97-3e2759876269 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:49:21.999300 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004202ea-2290-43d2-ad97-3e2759876269 None None] PATH_INFO: `/` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:49:22.000568 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 633/1274] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 1 14:49:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:49:22.006302 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4641233a-7cbd-48f4-bb5b-f8b2317cef99 None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:49:22.006717 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4641233a-7cbd-48f4-bb5b-f8b2317cef99 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:49:22.006856 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4641233a-7cbd-48f4-bb5b-f8b2317cef99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:49:22.024819 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-4641233a-7cbd-48f4-bb5b-f8b2317cef99 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:49:22.026014 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-4641233a-7cbd-48f4-bb5b-f8b2317cef99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:49:22.100888 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 642/1275] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 1 14:49:22 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 14:49:22.367386 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004202ea-2290-43d2-ad97-3e2759876269 None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:49:22.464341 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004202ea-2290-43d2-ad97-3e2759876269 None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:49:22.481256 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:49:22.497585 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f3d768-cde3-422e-9df0-c70536960f8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:49:22.498200 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f3d768-cde3-422e-9df0-c70536960f8d None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:49:22.498505 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f3d768-cde3-422e-9df0-c70536960f8d None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:49:22.498774 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f3d768-cde3-422e-9df0-c70536960f8d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:49:22.508504 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f3d768-cde3-422e-9df0-c70536960f8d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 14:49:22.513257 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application [None req-17f3d768-cde3-422e-9df0-c70536960f8d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 14:49:22.513257 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:49:22.513257 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:49:22.513257 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:49:22.513257 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:49:22.513257 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:49:22.513257 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:49:22.513257 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:49:22.513257 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:49:22.513257 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:49:22.513257 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:49:22.513257 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:49:22.513257 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 14:49:22.513257 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 14:49:22.513257 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 14:49:22.513257 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 14:49:22.514373 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 14:49:22.514373 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 14:49:22.514373 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 14:49:22.514373 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 14:49:22.514373 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:49:22.514373 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:49:22.514373 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:49:22.514373 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:49:22.514373 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:49:22.514373 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:49:22.514373 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:49:22.514373 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application with Lock( Sep 01 14:49:22.514373 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:49:22.514373 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:49:22.514373 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:49:22.514373 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:49:22.515257 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:49:22.515257 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:49:22.515257 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:49:22.515257 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:49:22.515257 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 14:49:22.515257 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 14:49:22.515257 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 14:49:22.515257 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 14:49:22.515257 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 14:49:22.515257 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 14:49:22.515257 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 14:49:22.515257 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Sep 01 14:49:22.515257 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 634/1276] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Sep 1 14:49:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:49:22.521002 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4641233a-7cbd-48f4-bb5b-f8b2317cef99 None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:49:22.599511 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:49:22.617967 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4641233a-7cbd-48f4-bb5b-f8b2317cef99 None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:49:22.650261 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d1c372-0250-499f-916c-50009c1c02a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:49:22.650841 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d1c372-0250-499f-916c-50009c1c02a9 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:49:22.651113 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d1c372-0250-499f-916c-50009c1c02a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:49:22.651379 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d1c372-0250-499f-916c-50009c1c02a9 None admin] PATH_INFO: `/v3/roles` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:49:22.652034 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d1c372-0250-499f-916c-50009c1c02a9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:49:22.652326 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d1c372-0250-499f-916c-50009c1c02a9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:49:22.653677 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d1c372-0250-499f-916c-50009c1c02a9 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:49:22.662472 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 643/1277] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Sep 1 14:49:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:49:22.668054 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f3d768-cde3-422e-9df0-c70536960f8d None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:49:22.765107 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f3d768-cde3-422e-9df0-c70536960f8d None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:49:22.798006 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75a0b8d-7583-44fe-a446-87189fda625c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:49:22.798649 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75a0b8d-7583-44fe-a446-87189fda625c None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:49:22.798930 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75a0b8d-7583-44fe-a446-87189fda625c None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:49:22.799200 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75a0b8d-7583-44fe-a446-87189fda625c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:49:22.808909 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75a0b8d-7583-44fe-a446-87189fda625c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:49:22.810395 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75a0b8d-7583-44fe-a446-87189fda625c None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:49:22.825956 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application [None req-b75a0b8d-7583-44fe-a446-87189fda625c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 14:49:22.825956 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:49:22.825956 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:49:22.825956 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:49:22.825956 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:49:22.825956 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:49:22.825956 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:49:22.825956 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:49:22.825956 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:49:22.825956 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:49:22.825956 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:49:22.825956 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:49:22.825956 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 14:49:22.825956 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 14:49:22.825956 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 14:49:22.825956 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 14:49:22.827095 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:49:22.827095 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:49:22.827095 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:49:22.827095 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:49:22.827095 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:49:22.827095 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:49:22.827095 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:49:22.827095 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application with Lock( Sep 01 14:49:22.827095 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:49:22.827095 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:49:22.827095 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:49:22.827095 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:49:22.827095 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:49:22.827095 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:49:22.827095 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:49:22.827095 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:49:22.827095 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 14:49:22.828255 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 14:49:22.828255 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 14:49:22.828255 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Sep 01 14:49:22.828255 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 635/1278] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 1 14:49:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:49:22.833522 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d1c372-0250-499f-916c-50009c1c02a9 None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:49:22.929105 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d1c372-0250-499f-916c-50009c1c02a9 None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:49:22.961929 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d521f2da-b318-4661-a5d9-244782a6f0f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:49:22.962537 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d521f2da-b318-4661-a5d9-244782a6f0f3 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:49:22.962825 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d521f2da-b318-4661-a5d9-244782a6f0f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:49:22.963096 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d521f2da-b318-4661-a5d9-244782a6f0f3 None admin] PATH_INFO: `/v3/domains` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:49:22.963732 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d521f2da-b318-4661-a5d9-244782a6f0f3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:49:22.964033 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d521f2da-b318-4661-a5d9-244782a6f0f3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:49:22.965383 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d521f2da-b318-4661-a5d9-244782a6f0f3 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:49:22.976736 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 644/1279] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Sep 1 14:49:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:49:22.982089 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75a0b8d-7583-44fe-a446-87189fda625c None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:49:23.011603 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:49:23.086375 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75a0b8d-7583-44fe-a446-87189fda625c None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:49:23.121110 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c30edeb-5d0d-41a4-b1ee-4b9808b986c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:49:23.121721 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c30edeb-5d0d-41a4-b1ee-4b9808b986c8 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:49:23.121997 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c30edeb-5d0d-41a4-b1ee-4b9808b986c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:49:23.122288 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c30edeb-5d0d-41a4-b1ee-4b9808b986c8 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:49:23.129932 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c30edeb-5d0d-41a4-b1ee-4b9808b986c8 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:49:23.131410 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c30edeb-5d0d-41a4-b1ee-4b9808b986c8 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:49:23.142204 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application [None req-5c30edeb-5d0d-41a4-b1ee-4b9808b986c8 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 01 14:49:23.142204 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:49:23.142204 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:49:23.142204 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:49:23.142204 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:49:23.142204 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:49:23.142204 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:49:23.142204 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:49:23.142204 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:49:23.142204 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:49:23.142204 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:49:23.142204 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:49:23.142204 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 14:49:23.142204 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 14:49:23.142204 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 14:49:23.142204 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 14:49:23.143378 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:49:23.143378 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:49:23.143378 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 14:49:23.143378 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 14:49:23.143378 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 14:49:23.143378 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 14:49:23.143378 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:49:23.143378 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:49:23.143378 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:49:23.143378 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:49:23.143378 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:49:23.143378 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application with Lock( Sep 01 14:49:23.143378 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:49:23.143378 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:49:23.143378 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:49:23.143378 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:49:23.143378 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:49:23.143378 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:49:23.144706 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:49:23.144706 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:49:23.144706 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 14:49:23.144706 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 14:49:23.144706 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 14:49:23.144706 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 14:49:23.144706 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 14:49:23.144706 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 14:49:23.144706 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 01 14:49:23.144706 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Sep 01 14:49:23.144706 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 636/1280] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 1 14:49:22 2022] GET /identity/v3/users/placement => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:49:23.149144 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d521f2da-b318-4661-a5d9-244782a6f0f3 None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:49:23.152838 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:49:23.246929 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d521f2da-b318-4661-a5d9-244782a6f0f3 None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:49:23.279177 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1cbf62-4b4d-4367-893a-0fabaf924fa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:49:23.279764 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1cbf62-4b4d-4367-893a-0fabaf924fa0 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:49:23.280047 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1cbf62-4b4d-4367-893a-0fabaf924fa0 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:49:23.280316 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1cbf62-4b4d-4367-893a-0fabaf924fa0 None admin] PATH_INFO: `/v3/users` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:49:23.281031 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1cbf62-4b4d-4367-893a-0fabaf924fa0 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:49:23.281333 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1cbf62-4b4d-4367-893a-0fabaf924fa0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:49:23.282773 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1cbf62-4b4d-4367-893a-0fabaf924fa0 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:49:23.299580 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 645/1281] 10.222.0.33 () {56 vars in 1143 bytes} [Thu Sep 1 14:49:23 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:49:23.305126 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c30edeb-5d0d-41a4-b1ee-4b9808b986c8 None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:49:23.393027 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:49:23.402658 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c30edeb-5d0d-41a4-b1ee-4b9808b986c8 None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:49:23.434647 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b064c23-0e13-432e-911c-3ff7bf8913d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:49:23.435229 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b064c23-0e13-432e-911c-3ff7bf8913d9 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:49:23.435506 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b064c23-0e13-432e-911c-3ff7bf8913d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:49:23.435776 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b064c23-0e13-432e-911c-3ff7bf8913d9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:49:23.501037 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b064c23-0e13-432e-911c-3ff7bf8913d9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:49:23.503031 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b064c23-0e13-432e-911c-3ff7bf8913d9 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:49:23.587055 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:49:23.597844 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 637/1282] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Sep 1 14:49:23 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 14:49:23.603704 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1cbf62-4b4d-4367-893a-0fabaf924fa0 None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:49:23.661888 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:49:23.699939 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1cbf62-4b4d-4367-893a-0fabaf924fa0 None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:49:23.732622 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e392d40-b336-4505-95a8-5cf78183f12a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:49:23.733239 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e392d40-b336-4505-95a8-5cf78183f12a None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:49:23.733516 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e392d40-b336-4505-95a8-5cf78183f12a None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:49:23.733783 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e392d40-b336-4505-95a8-5cf78183f12a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:49:23.744065 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e392d40-b336-4505-95a8-5cf78183f12a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:49:23.745498 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e392d40-b336-4505-95a8-5cf78183f12a None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:49:23.759570 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application [None req-2e392d40-b336-4505-95a8-5cf78183f12a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 14:49:23.759570 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:49:23.759570 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:49:23.759570 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:49:23.759570 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:49:23.759570 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:49:23.759570 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:49:23.759570 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:49:23.759570 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:49:23.759570 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:49:23.759570 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:49:23.759570 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:49:23.759570 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 14:49:23.759570 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 14:49:23.759570 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 14:49:23.759570 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 14:49:23.760716 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:49:23.760716 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:49:23.760716 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:49:23.760716 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:49:23.760716 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:49:23.760716 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:49:23.760716 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:49:23.760716 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application with Lock( Sep 01 14:49:23.760716 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:49:23.760716 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:49:23.760716 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:49:23.760716 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:49:23.760716 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:49:23.760716 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:49:23.760716 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:49:23.760716 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:49:23.760716 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 14:49:23.761856 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 14:49:23.761856 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 14:49:23.761856 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Sep 01 14:49:23.761856 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 646/1283] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 1 14:49:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:49:23.766318 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b064c23-0e13-432e-911c-3ff7bf8913d9 None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:49:23.859761 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b064c23-0e13-432e-911c-3ff7bf8913d9 None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:49:23.892002 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2880d8-5198-4fdf-864d-e7b82b598810 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:49:23.892579 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2880d8-5198-4fdf-864d-e7b82b598810 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:49:23.893368 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2880d8-5198-4fdf-864d-e7b82b598810 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:49:23.893634 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2880d8-5198-4fdf-864d-e7b82b598810 None admin] PATH_INFO: `/v3/domains` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:49:23.894277 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2880d8-5198-4fdf-864d-e7b82b598810 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81341) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:49:23.894583 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2880d8-5198-4fdf-864d-e7b82b598810 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:49:23.895937 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2880d8-5198-4fdf-864d-e7b82b598810 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:49:23.906618 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 638/1284] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Sep 1 14:49:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:49:23.912560 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e392d40-b336-4505-95a8-5cf78183f12a None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:49:24.009035 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e392d40-b336-4505-95a8-5cf78183f12a None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:49:24.022588 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:49:24.043709 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72527c7-858c-4662-b760-c3580a1e243e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:49:24.044368 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72527c7-858c-4662-b760-c3580a1e243e None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:49:24.044600 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72527c7-858c-4662-b760-c3580a1e243e None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:49:24.044889 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72527c7-858c-4662-b760-c3580a1e243e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:49:24.052582 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72527c7-858c-4662-b760-c3580a1e243e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:49:24.054020 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72527c7-858c-4662-b760-c3580a1e243e None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:49:24.066022 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application [None req-b72527c7-858c-4662-b760-c3580a1e243e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 14:49:24.066022 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:49:24.066022 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:49:24.066022 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:49:24.066022 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:49:24.066022 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:49:24.066022 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:49:24.066022 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:49:24.066022 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:49:24.066022 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:49:24.066022 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:49:24.066022 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:49:24.066022 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 14:49:24.066022 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 14:49:24.066022 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 14:49:24.067120 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 14:49:24.067120 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:49:24.067120 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:49:24.067120 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:49:24.067120 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:49:24.067120 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:49:24.067120 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:49:24.067120 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:49:24.067120 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application with Lock( Sep 01 14:49:24.067120 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:49:24.067120 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:49:24.067120 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:49:24.067120 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:49:24.067120 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:49:24.067120 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:49:24.067120 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:49:24.067120 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:49:24.068368 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 14:49:24.068368 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 14:49:24.068368 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 14:49:24.068368 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 14:49:24.068368 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 14:49:24.068368 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 14:49:24.068368 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 14:49:24.068368 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: ERROR keystone.server.flask.application Sep 01 14:49:24.068368 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 647/1285] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Sep 1 14:49:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:49:24.073227 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2880d8-5198-4fdf-864d-e7b82b598810 None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:49:24.122036 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:49:24.172366 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2880d8-5198-4fdf-864d-e7b82b598810 None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:49:24.204056 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc515d9-3f11-48b5-bc06-9035e7030852 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:49:24.204640 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc515d9-3f11-48b5-bc06-9035e7030852 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:49:24.204913 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc515d9-3f11-48b5-bc06-9035e7030852 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:49:24.205177 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc515d9-3f11-48b5-bc06-9035e7030852 None admin] PATH_INFO: `/v3/projects` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:49:24.206402 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc515d9-3f11-48b5-bc06-9035e7030852 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81341) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:49:24.206698 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc515d9-3f11-48b5-bc06-9035e7030852 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:49:24.208119 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc515d9-3f11-48b5-bc06-9035e7030852 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:49:24.219461 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 639/1286] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Sep 1 14:49:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:49:24.226055 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72527c7-858c-4662-b760-c3580a1e243e None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:49:24.326193 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72527c7-858c-4662-b760-c3580a1e243e None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:49:24.358685 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc657fb3-df5a-4b0c-8595-4c009f299c27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:49:24.359281 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc657fb3-df5a-4b0c-8595-4c009f299c27 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:49:24.359559 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc657fb3-df5a-4b0c-8595-4c009f299c27 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:49:24.359827 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc657fb3-df5a-4b0c-8595-4c009f299c27 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:49:24.360548 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc657fb3-df5a-4b0c-8595-4c009f299c27 None admin] RBAC: Adding query filter params (role.id=2a2adb769e55476b82e5014897e0a9c0, scope.project.id=e8c85b4bd17e4e89b56bf0f831b474d5, user.id=7f312c8dcabe48ef9ea453364ade38ff) {{(pid=81342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:49:24.360851 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc657fb3-df5a-4b0c-8595-4c009f299c27 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:49:24.362310 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc657fb3-df5a-4b0c-8595-4c009f299c27 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:49:24.367856 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 648/1287] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Sep 1 14:49:24 2022] GET /identity/v3/role_assignments?user.id=7f312c8dcabe48ef9ea453364ade38ff&scope.project.id=e8c85b4bd17e4e89b56bf0f831b474d5&role.id=2a2adb769e55476b82e5014897e0a9c0 => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:49:26.265466 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687ffa22-16ec-4175-ab7e-74d5dd455d5e None None] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:49:26.266030 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687ffa22-16ec-4175-ab7e-74d5dd455d5e None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:49:26.266030 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687ffa22-16ec-4175-ab7e-74d5dd455d5e None None] PATH_INFO: `/` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:49:26.267358 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 640/1288] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 1 14:49:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:49:26.272371 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691f9d8d-73b0-48f5-884a-23ae6910e28c None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:49:26.272810 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691f9d8d-73b0-48f5-884a-23ae6910e28c None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:49:26.272911 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691f9d8d-73b0-48f5-884a-23ae6910e28c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:49:26.290713 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-691f9d8d-73b0-48f5-884a-23ae6910e28c None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:49:26.292162 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-691f9d8d-73b0-48f5-884a-23ae6910e28c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:49:26.323352 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:49:26.364079 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 649/1289] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 1 14:49:26 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 14:49:26.629956 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687ffa22-16ec-4175-ab7e-74d5dd455d5e None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:49:26.730103 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687ffa22-16ec-4175-ab7e-74d5dd455d5e None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:49:26.740970 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:49:26.763671 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810ac15f-3654-4cba-9de3-877e977c21c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:49:26.764287 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810ac15f-3654-4cba-9de3-877e977c21c3 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:49:26.764562 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810ac15f-3654-4cba-9de3-877e977c21c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:49:26.764830 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810ac15f-3654-4cba-9de3-877e977c21c3 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:49:26.765399 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810ac15f-3654-4cba-9de3-877e977c21c3 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:49:26.766812 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810ac15f-3654-4cba-9de3-877e977c21c3 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:49:26.775394 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application [None req-810ac15f-3654-4cba-9de3-877e977c21c3 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 01 14:49:26.775394 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:49:26.775394 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 14:49:26.775394 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 14:49:26.775394 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 14:49:26.775394 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 14:49:26.775394 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 14:49:26.775394 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 14:49:26.775394 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 01 14:49:26.775394 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Sep 01 14:49:26.775394 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 14:49:26.775394 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Sep 01 14:49:26.775394 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:49:26.775394 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:49:26.775394 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:49:26.775394 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:49:26.775394 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:49:26.776663 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:49:26.776663 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:49:26.776663 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:49:26.776663 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:49:26.776663 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:49:26.776663 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:49:26.776663 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 14:49:26.776663 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 14:49:26.776663 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 14:49:26.776663 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 14:49:26.776663 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:49:26.776663 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:49:26.776663 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:49:26.776663 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:49:26.776663 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:49:26.776663 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:49:26.777645 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:49:26.777645 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application with Lock( Sep 01 14:49:26.777645 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:49:26.777645 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:49:26.777645 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:49:26.777645 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:49:26.777645 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:49:26.777645 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:49:26.777645 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:49:26.777645 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:49:26.777645 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 14:49:26.777645 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 14:49:26.777645 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 01 14:49:26.777645 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Sep 01 14:49:26.777645 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 641/1290] 10.222.0.33 () {56 vars in 1117 bytes} [Thu Sep 1 14:49:26 2022] GET /identity/v3/services/placement => generated 90 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:49:26.782438 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691f9d8d-73b0-48f5-884a-23ae6910e28c None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:49:26.877562 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691f9d8d-73b0-48f5-884a-23ae6910e28c None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:49:26.910078 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a4557e-eadd-4af4-84ff-444fa4bb26ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:49:26.910637 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a4557e-eadd-4af4-84ff-444fa4bb26ef None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:49:26.910916 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a4557e-eadd-4af4-84ff-444fa4bb26ef None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:49:26.911185 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a4557e-eadd-4af4-84ff-444fa4bb26ef None admin] PATH_INFO: `/v3/services` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:49:26.911812 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a4557e-eadd-4af4-84ff-444fa4bb26ef None admin] RBAC: Adding query filter params (name=placement) {{(pid=81342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:49:26.912110 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a4557e-eadd-4af4-84ff-444fa4bb26ef None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:49:26.913473 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a4557e-eadd-4af4-84ff-444fa4bb26ef None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:49:26.918055 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 650/1291] 10.222.0.33 () {56 vars in 1116 bytes} [Thu Sep 1 14:49:26 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:49:26.923434 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810ac15f-3654-4cba-9de3-877e977c21c3 None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:49:27.019427 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810ac15f-3654-4cba-9de3-877e977c21c3 None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:49:27.053514 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8208c9-ee35-4640-9f91-9dc74091df4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:49:27.054140 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8208c9-ee35-4640-9f91-9dc74091df4f None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:49:27.054551 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8208c9-ee35-4640-9f91-9dc74091df4f None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:49:27.054800 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8208c9-ee35-4640-9f91-9dc74091df4f None admin] PATH_INFO: `/v3/services` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:49:27.055561 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8208c9-ee35-4640-9f91-9dc74091df4f None admin] RBAC: Adding query filter params (type=placement) {{(pid=81341) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:49:27.055857 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8208c9-ee35-4640-9f91-9dc74091df4f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:49:27.057469 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8208c9-ee35-4640-9f91-9dc74091df4f None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:49:27.062564 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 642/1292] 10.222.0.33 () {56 vars in 1116 bytes} [Thu Sep 1 14:49:26 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:49:28.946814 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde91219-d3e3-49c1-8223-dc4db1df3f46 None None] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:49:28.947327 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde91219-d3e3-49c1-8223-dc4db1df3f46 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:49:28.947327 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde91219-d3e3-49c1-8223-dc4db1df3f46 None None] PATH_INFO: `/` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:49:28.948601 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 651/1293] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 1 14:49:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:49:28.953559 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d200bba-27e0-4380-ac94-acfcc6532885 None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:49:28.953984 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d200bba-27e0-4380-ac94-acfcc6532885 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:49:28.954102 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d200bba-27e0-4380-ac94-acfcc6532885 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:49:28.972708 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-7d200bba-27e0-4380-ac94-acfcc6532885 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:49:28.973895 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-7d200bba-27e0-4380-ac94-acfcc6532885 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:49:29.004420 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:49:29.044429 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 643/1294] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 1 14:49:28 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 14:49:29.325714 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde91219-d3e3-49c1-8223-dc4db1df3f46 None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:49:29.357340 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:49:29.425999 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde91219-d3e3-49c1-8223-dc4db1df3f46 None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:49:29.458834 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ad794d-98ab-42a4-ba57-e7c96f341836 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:49:29.459429 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ad794d-98ab-42a4-ba57-e7c96f341836 None admin] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:49:29.459709 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ad794d-98ab-42a4-ba57-e7c96f341836 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:49:29.459982 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ad794d-98ab-42a4-ba57-e7c96f341836 None admin] PATH_INFO: `/v3/services` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:49:29.460632 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ad794d-98ab-42a4-ba57-e7c96f341836 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:49:29.461999 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ad794d-98ab-42a4-ba57-e7c96f341836 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:49:29.476584 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 652/1295] 10.222.0.33 () {60 vars in 1141 bytes} [Thu Sep 1 14:49:29 2022] POST /identity/v3/services => generated 246 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:49:31.396417 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6a5423-dab7-46ec-9ada-9ac1b31bf361 None None] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:49:31.396967 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6a5423-dab7-46ec-9ada-9ac1b31bf361 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:49:31.396967 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6a5423-dab7-46ec-9ada-9ac1b31bf361 None None] PATH_INFO: `/` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:49:31.398313 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 644/1296] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 1 14:49:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:49:31.403416 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd6d18b-70fe-4528-bcf2-25cf5466193c None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:49:31.403845 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd6d18b-70fe-4528-bcf2-25cf5466193c None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:49:31.404058 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd6d18b-70fe-4528-bcf2-25cf5466193c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:49:31.422218 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-3cd6d18b-70fe-4528-bcf2-25cf5466193c None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:49:31.423410 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-3cd6d18b-70fe-4528-bcf2-25cf5466193c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:49:31.493629 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 653/1297] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 1 14:49:31 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 14:49:31.761085 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6a5423-dab7-46ec-9ada-9ac1b31bf361 None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:49:31.858206 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6a5423-dab7-46ec-9ada-9ac1b31bf361 None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:49:31.890176 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aed5021-9d61-43fd-92d0-7f0470ad637c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:49:31.890807 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aed5021-9d61-43fd-92d0-7f0470ad637c None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:49:31.891080 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aed5021-9d61-43fd-92d0-7f0470ad637c None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:49:31.891344 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aed5021-9d61-43fd-92d0-7f0470ad637c None admin] PATH_INFO: `/v3/services/placement` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:49:31.891910 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aed5021-9d61-43fd-92d0-7f0470ad637c None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:49:31.893875 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aed5021-9d61-43fd-92d0-7f0470ad637c None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:49:31.902294 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application [None req-7aed5021-9d61-43fd-92d0-7f0470ad637c None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 01 14:49:31.902294 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:49:31.902294 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 14:49:31.902294 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 14:49:31.902294 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 14:49:31.902294 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 14:49:31.902294 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 14:49:31.902294 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 14:49:31.902294 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 01 14:49:31.902294 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Sep 01 14:49:31.902294 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 14:49:31.902294 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Sep 01 14:49:31.902294 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:49:31.902294 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:49:31.902294 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:49:31.902294 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:49:31.902294 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:49:31.903497 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:49:31.903497 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:49:31.903497 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:49:31.903497 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:49:31.903497 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:49:31.903497 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:49:31.903497 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 14:49:31.903497 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 14:49:31.903497 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 14:49:31.903497 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 14:49:31.903497 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:49:31.903497 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:49:31.903497 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:49:31.903497 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:49:31.903497 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:49:31.903497 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:49:31.904557 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:49:31.904557 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application with Lock( Sep 01 14:49:31.904557 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:49:31.904557 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:49:31.904557 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:49:31.904557 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:49:31.904557 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:49:31.904557 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:49:31.904557 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:49:31.904557 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:49:31.904557 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 14:49:31.904557 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 14:49:31.904557 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 01 14:49:31.904557 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Sep 01 14:49:31.904557 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 645/1298] 10.222.0.33 () {56 vars in 1117 bytes} [Thu Sep 1 14:49:31 2022] GET /identity/v3/services/placement => generated 90 bytes in 143 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:49:31.909169 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd6d18b-70fe-4528-bcf2-25cf5466193c None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:49:31.927004 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:49:32.005162 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd6d18b-70fe-4528-bcf2-25cf5466193c None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:49:32.037465 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a31d3ed-157b-4eee-ac23-cf61bf46fa63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:49:32.038056 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a31d3ed-157b-4eee-ac23-cf61bf46fa63 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:49:32.038361 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a31d3ed-157b-4eee-ac23-cf61bf46fa63 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:49:32.038630 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a31d3ed-157b-4eee-ac23-cf61bf46fa63 None admin] PATH_INFO: `/v3/services` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:49:32.039249 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a31d3ed-157b-4eee-ac23-cf61bf46fa63 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:49:32.039546 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a31d3ed-157b-4eee-ac23-cf61bf46fa63 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:49:32.040902 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a31d3ed-157b-4eee-ac23-cf61bf46fa63 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:49:32.046272 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 654/1299] 10.222.0.33 () {56 vars in 1116 bytes} [Thu Sep 1 14:49:31 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:49:32.053017 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aed5021-9d61-43fd-92d0-7f0470ad637c None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:49:32.088217 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:49:32.159106 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aed5021-9d61-43fd-92d0-7f0470ad637c None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:49:32.191549 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0247d41-72cd-4935-b9ca-fa2a6cddf1f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:49:32.192081 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0247d41-72cd-4935-b9ca-fa2a6cddf1f3 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:49:32.192353 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0247d41-72cd-4935-b9ca-fa2a6cddf1f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:49:32.192619 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0247d41-72cd-4935-b9ca-fa2a6cddf1f3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:49:32.193573 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0247d41-72cd-4935-b9ca-fa2a6cddf1f3 None admin] RBAC: Adding query filter params (interface=public, service_id=6643c36f709e4103b25a8e1d911a128a, region_id=RegionOne) {{(pid=81341) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:49:32.193871 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0247d41-72cd-4935-b9ca-fa2a6cddf1f3 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:49:32.195267 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0247d41-72cd-4935-b9ca-fa2a6cddf1f3 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:49:32.200031 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 646/1300] 10.222.0.33 () {56 vars in 1251 bytes} [Thu Sep 1 14:49:32 2022] GET /identity/v3/endpoints?service_id=6643c36f709e4103b25a8e1d911a128a&interface=public®ion_id=RegionOne => generated 194 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 14:49:32.205747 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a31d3ed-157b-4eee-ac23-cf61bf46fa63 None admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:49:32.289287 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:49:32.307452 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a31d3ed-157b-4eee-ac23-cf61bf46fa63 None admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:49:32.339992 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbe33e3-03b9-4741-afd2-1db689d2a214 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:49:32.340583 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbe33e3-03b9-4741-afd2-1db689d2a214 None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:49:32.340864 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbe33e3-03b9-4741-afd2-1db689d2a214 None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:49:32.341133 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbe33e3-03b9-4741-afd2-1db689d2a214 None admin] PATH_INFO: `/v3/services` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:49:32.341731 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbe33e3-03b9-4741-afd2-1db689d2a214 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:49:32.343126 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbe33e3-03b9-4741-afd2-1db689d2a214 None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:49:32.349697 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 655/1301] 10.222.0.33 () {56 vars in 1087 bytes} [Thu Sep 1 14:49:32 2022] GET /identity/v3/services => generated 2191 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:49:34.229521 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf3cd02-aed9-4de2-94ab-97c9df8be275 None None] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:49:34.230049 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf3cd02-aed9-4de2-94ab-97c9df8be275 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:49:34.230049 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf3cd02-aed9-4de2-94ab-97c9df8be275 None None] PATH_INFO: `/` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:49:34.231418 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 647/1302] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 1 14:49:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:49:34.236504 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94355a8b-1d2d-41e9-8194-dfa6316d0fd1 None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:49:34.236940 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94355a8b-1d2d-41e9-8194-dfa6316d0fd1 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:49:34.237033 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94355a8b-1d2d-41e9-8194-dfa6316d0fd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:49:34.255491 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-94355a8b-1d2d-41e9-8194-dfa6316d0fd1 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:49:34.256665 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-94355a8b-1d2d-41e9-8194-dfa6316d0fd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:49:34.320269 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 656/1303] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 1 14:49:34 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 14:49:34.589375 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf3cd02-aed9-4de2-94ab-97c9df8be275 None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:49:34.682156 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:49:34.689189 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf3cd02-aed9-4de2-94ab-97c9df8be275 None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:49:34.722127 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596db678-ec54-4556-96f7-399b75417957 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:49:34.722759 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596db678-ec54-4556-96f7-399b75417957 None admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:49:34.723034 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596db678-ec54-4556-96f7-399b75417957 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:49:34.723300 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596db678-ec54-4556-96f7-399b75417957 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:49:34.723866 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596db678-ec54-4556-96f7-399b75417957 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:49:34.725235 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596db678-ec54-4556-96f7-399b75417957 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:49:34.733634 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application [None req-596db678-ec54-4556-96f7-399b75417957 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 01 14:49:34.733634 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:49:34.733634 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 14:49:34.733634 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 14:49:34.733634 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 14:49:34.733634 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 14:49:34.733634 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 14:49:34.733634 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 14:49:34.733634 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 01 14:49:34.733634 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Sep 01 14:49:34.733634 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 14:49:34.733634 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Sep 01 14:49:34.733634 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 14:49:34.733634 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 14:49:34.733634 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 14:49:34.733634 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 14:49:34.733634 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 14:49:34.734841 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 14:49:34.734841 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 14:49:34.734841 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 14:49:34.734841 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 14:49:34.734841 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 14:49:34.734841 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 14:49:34.734841 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 14:49:34.734841 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 14:49:34.734841 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 14:49:34.734841 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 14:49:34.734841 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 14:49:34.734841 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 14:49:34.734841 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 14:49:34.734841 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 14:49:34.734841 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 14:49:34.734841 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 14:49:34.736073 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 14:49:34.736073 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application with Lock( Sep 01 14:49:34.736073 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 14:49:34.736073 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self._enter() Sep 01 14:49:34.736073 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 14:49:34.736073 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 14:49:34.736073 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 14:49:34.736073 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application return self.creator() Sep 01 14:49:34.736073 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 14:49:34.736073 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application created_value = creator( Sep 01 14:49:34.736073 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 14:49:34.736073 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 14:49:34.736073 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 01 14:49:34.736073 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: ERROR keystone.server.flask.application Sep 01 14:49:34.736073 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 648/1304] 10.222.0.33 () {56 vars in 1117 bytes} [Thu Sep 1 14:49:34 2022] GET /identity/v3/services/placement => generated 90 bytes in 146 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 14:49:34.740638 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94355a8b-1d2d-41e9-8194-dfa6316d0fd1 None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:49:34.812158 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:49:34.835185 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94355a8b-1d2d-41e9-8194-dfa6316d0fd1 None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:49:34.866780 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab61a68-d021-4dfe-a509-cf381b37701e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:49:34.867311 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab61a68-d021-4dfe-a509-cf381b37701e None admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:49:34.867582 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab61a68-d021-4dfe-a509-cf381b37701e None admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:49:34.867845 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab61a68-d021-4dfe-a509-cf381b37701e None admin] PATH_INFO: `/v3/services` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:49:34.868459 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab61a68-d021-4dfe-a509-cf381b37701e None admin] RBAC: Adding query filter params (name=placement) {{(pid=81342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:49:34.868752 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab61a68-d021-4dfe-a509-cf381b37701e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:49:34.870097 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab61a68-d021-4dfe-a509-cf381b37701e None admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:49:34.874982 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 657/1305] 10.222.0.33 () {56 vars in 1116 bytes} [Thu Sep 1 14:49:34 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:49:34.881197 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596db678-ec54-4556-96f7-399b75417957 None admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:49:34.976580 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596db678-ec54-4556-96f7-399b75417957 None admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:49:35.008917 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d41973-7cfa-49bf-b4eb-5865ed006153 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:49:35.009495 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d41973-7cfa-49bf-b4eb-5865ed006153 None admin] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:49:35.009770 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d41973-7cfa-49bf-b4eb-5865ed006153 None admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:49:35.010034 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d41973-7cfa-49bf-b4eb-5865ed006153 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:49:35.010709 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d41973-7cfa-49bf-b4eb-5865ed006153 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:49:35.012070 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d41973-7cfa-49bf-b4eb-5865ed006153 None admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:49:35.031460 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 649/1306] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Sep 1 14:49:34 2022] POST /identity/v3/endpoints => generated 331 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:49:44.872531 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=30 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:49:45.027808 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:50:37.776368 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4032f39-bb09-450f-bc19-5dfbf4aeddca None None] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:50:37.776945 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4032f39-bb09-450f-bc19-5dfbf4aeddca None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:50:37.776945 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4032f39-bb09-450f-bc19-5dfbf4aeddca None None] PATH_INFO: `/` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:50:37.783665 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77b46e7-3dda-4a93-997b-87e3243ef9b6 None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:50:37.784147 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77b46e7-3dda-4a93-997b-87e3243ef9b6 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:50:37.784239 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77b46e7-3dda-4a93-997b-87e3243ef9b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:50:37.788978 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 658/1307] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 1 14:50:37 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:50:37.876854 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-c77b46e7-3dda-4a93-997b-87e3243ef9b6 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:50:37.878222 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-c77b46e7-3dda-4a93-997b-87e3243ef9b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:50:37.997829 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 650/1308] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 1 14:50:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 14:50:38.321320 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3896c8fb-bcdc-4592-bdeb-9ff3d0dacac2 None None] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:50:38.321858 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3896c8fb-bcdc-4592-bdeb-9ff3d0dacac2 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:50:38.322355 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3896c8fb-bcdc-4592-bdeb-9ff3d0dacac2 None None] PATH_INFO: `/` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:50:38.324145 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 659/1309] 10.222.0.33 () {54 vars in 965 bytes} [Thu Sep 1 14:50:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:50:38.330053 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e692ec-075b-4c90-8e27-4f0c36be856c None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:50:38.330375 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e692ec-075b-4c90-8e27-4f0c36be856c None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:50:38.330653 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e692ec-075b-4c90-8e27-4f0c36be856c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:50:38.412793 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-43e692ec-075b-4c90-8e27-4f0c36be856c None None] MFA Rules not processed for user `2790149639aa418bbc4e4982e93eda44`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:50:38.414007 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-43e692ec-075b-4c90-8e27-4f0c36be856c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:50:38.437292 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:50:38.499478 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 651/1310] 10.222.0.33 () {58 vars in 1062 bytes} [Thu Sep 1 14:50:38 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:50:38.508316 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3896c8fb-bcdc-4592-bdeb-9ff3d0dacac2 None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:50:38.510058 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-3896c8fb-bcdc-4592-bdeb-9ff3d0dacac2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:50:38.645268 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3896c8fb-bcdc-4592-bdeb-9ff3d0dacac2 None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:50:38.734354 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4d7949-914a-45e4-abc7-fe32e8a85504 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2790149639aa418bbc4e4982e93eda44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:50:38.735601 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4d7949-914a-45e4-abc7-fe32e8a85504 service neutron] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:50:38.735892 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4d7949-914a-45e4-abc7-fe32e8a85504 service neutron] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:50:38.736160 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4d7949-914a-45e4-abc7-fe32e8a85504 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:50:38.737981 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-8d4d7949-914a-45e4-abc7-fe32e8a85504 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:50:38.821648 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:50:38.899048 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4d7949-914a-45e4-abc7-fe32e8a85504 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:50:38.900614 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4d7949-914a-45e4-abc7-fe32e8a85504 service neutron] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:50:38.922085 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 660/1311] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 14:50:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:50:41.162312 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6d41ae-a03d-424f-9fa2-011a8a272706 None None] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:50:41.162899 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6d41ae-a03d-424f-9fa2-011a8a272706 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:50:41.162899 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6d41ae-a03d-424f-9fa2-011a8a272706 None None] PATH_INFO: `/` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:50:41.164210 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 652/1312] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 1 14:50:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:50:41.169742 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4d0b68-629f-4937-83f2-002cce5dc81c None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:50:41.170254 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4d0b68-629f-4937-83f2-002cce5dc81c None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:50:41.170385 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4d0b68-629f-4937-83f2-002cce5dc81c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:50:41.257894 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-3f4d0b68-629f-4937-83f2-002cce5dc81c None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:50:41.259218 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-3f4d0b68-629f-4937-83f2-002cce5dc81c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:50:41.357343 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 661/1313] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 1 14:50:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:50:41.643715 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6d41ae-a03d-424f-9fa2-011a8a272706 None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:50:41.645257 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-9a6d41ae-a03d-424f-9fa2-011a8a272706 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:50:41.746512 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6d41ae-a03d-424f-9fa2-011a8a272706 None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:50:41.814956 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:50:41.817677 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b1abb8-1a52-45c4-b25f-e795c5f1d49c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2790149639aa418bbc4e4982e93eda44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:50:41.818516 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b1abb8-1a52-45c4-b25f-e795c5f1d49c service neutron] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:50:41.818967 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b1abb8-1a52-45c4-b25f-e795c5f1d49c service neutron] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:50:41.819425 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b1abb8-1a52-45c4-b25f-e795c5f1d49c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:50:41.821057 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-d9b1abb8-1a52-45c4-b25f-e795c5f1d49c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:50:41.926324 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b1abb8-1a52-45c4-b25f-e795c5f1d49c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:50:41.927774 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b1abb8-1a52-45c4-b25f-e795c5f1d49c service neutron] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:50:41.946620 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 653/1314] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 14:50:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:50:44.133778 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58674df3-6b96-4f2d-8faf-230093c5c836 None None] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:50:44.134371 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58674df3-6b96-4f2d-8faf-230093c5c836 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:50:44.134371 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58674df3-6b96-4f2d-8faf-230093c5c836 None None] PATH_INFO: `/` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:50:44.135695 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 662/1315] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 1 14:50:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:50:44.140794 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b9b34f-37b8-464f-a508-363e351c2015 None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:50:44.141723 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b9b34f-37b8-464f-a508-363e351c2015 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:50:44.141816 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b9b34f-37b8-464f-a508-363e351c2015 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:50:44.205400 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-01b9b34f-37b8-464f-a508-363e351c2015 None None] MFA Rules not processed for user `4b65298713a24cada6ba75d786076e93`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:50:44.206655 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-01b9b34f-37b8-464f-a508-363e351c2015 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:50:44.256484 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:50:44.301445 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 654/1316] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 1 14:50:44 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:50:44.586168 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58674df3-6b96-4f2d-8faf-230093c5c836 None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:50:44.587767 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-58674df3-6b96-4f2d-8faf-230093c5c836 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:50:44.597400 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:50:44.692237 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58674df3-6b96-4f2d-8faf-230093c5c836 None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:50:44.763088 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3d7c5e-acf3-41de-b99a-06cd9b63a0eb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2790149639aa418bbc4e4982e93eda44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:50:44.763682 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3d7c5e-acf3-41de-b99a-06cd9b63a0eb service neutron] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:50:44.763953 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3d7c5e-acf3-41de-b99a-06cd9b63a0eb service neutron] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:50:44.764216 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3d7c5e-acf3-41de-b99a-06cd9b63a0eb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:50:44.765773 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-fd3d7c5e-acf3-41de-b99a-06cd9b63a0eb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:50:44.825588 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:50:44.871686 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3d7c5e-acf3-41de-b99a-06cd9b63a0eb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:50:44.873046 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3d7c5e-acf3-41de-b99a-06cd9b63a0eb service neutron] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:50:44.891478 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 663/1317] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 14:50:44 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:50:47.314054 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e19ede-fea2-4a91-b1bf-420f7f3d44a5 None None] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:50:47.314630 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e19ede-fea2-4a91-b1bf-420f7f3d44a5 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:50:47.314630 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e19ede-fea2-4a91-b1bf-420f7f3d44a5 None None] PATH_INFO: `/` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:50:47.315974 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 655/1318] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 1 14:50:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:50:47.321305 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0674976c-f98f-4399-8652-70e99fe1321d None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:50:47.321741 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0674976c-f98f-4399-8652-70e99fe1321d None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:50:47.321845 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0674976c-f98f-4399-8652-70e99fe1321d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:50:47.386464 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-0674976c-f98f-4399-8652-70e99fe1321d None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:50:47.387672 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-0674976c-f98f-4399-8652-70e99fe1321d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:50:47.482180 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 664/1319] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 1 14:50:47 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:50:47.765810 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e19ede-fea2-4a91-b1bf-420f7f3d44a5 None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:50:47.767909 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-f7e19ede-fea2-4a91-b1bf-420f7f3d44a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:50:47.868076 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e19ede-fea2-4a91-b1bf-420f7f3d44a5 None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:50:47.938610 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba77701f-aa15-4d6e-97be-665c35256b9d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2790149639aa418bbc4e4982e93eda44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:50:47.939150 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba77701f-aa15-4d6e-97be-665c35256b9d service neutron] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:50:47.939432 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba77701f-aa15-4d6e-97be-665c35256b9d service neutron] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:50:47.939705 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba77701f-aa15-4d6e-97be-665c35256b9d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:50:47.941299 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-ba77701f-aa15-4d6e-97be-665c35256b9d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:50:47.948905 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:50:48.048335 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba77701f-aa15-4d6e-97be-665c35256b9d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:50:48.049779 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba77701f-aa15-4d6e-97be-665c35256b9d service neutron] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:50:48.068219 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 656/1320] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 14:50:47 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:50:50.601421 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47081bcf-1769-46c1-bcbf-ad8d2465dda5 None None] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:50:50.601964 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47081bcf-1769-46c1-bcbf-ad8d2465dda5 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:50:50.601964 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47081bcf-1769-46c1-bcbf-ad8d2465dda5 None None] PATH_INFO: `/` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:50:50.603392 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 665/1321] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 1 14:50:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:50:50.608617 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ab5bff-8a82-408b-a960-3e5221990189 None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:50:50.609032 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ab5bff-8a82-408b-a960-3e5221990189 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:50:50.609286 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ab5bff-8a82-408b-a960-3e5221990189 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:50:50.675601 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-29ab5bff-8a82-408b-a960-3e5221990189 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:50:50.676820 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-29ab5bff-8a82-408b-a960-3e5221990189 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:50:50.733646 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:50:50.775240 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 657/1322] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 1 14:50:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:50:51.058971 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47081bcf-1769-46c1-bcbf-ad8d2465dda5 None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:50:51.060506 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-47081bcf-1769-46c1-bcbf-ad8d2465dda5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:50:51.080465 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:50:51.165912 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47081bcf-1769-46c1-bcbf-ad8d2465dda5 None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:50:51.239432 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ec0da0-616a-41e7-b1b6-d0ff4dc43a09 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2790149639aa418bbc4e4982e93eda44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:50:51.239931 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ec0da0-616a-41e7-b1b6-d0ff4dc43a09 service neutron] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:50:51.240213 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ec0da0-616a-41e7-b1b6-d0ff4dc43a09 service neutron] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:50:51.240470 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ec0da0-616a-41e7-b1b6-d0ff4dc43a09 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:50:51.242080 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-61ec0da0-616a-41e7-b1b6-d0ff4dc43a09 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:50:51.313036 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:50:51.351705 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ec0da0-616a-41e7-b1b6-d0ff4dc43a09 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:50:51.353150 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ec0da0-616a-41e7-b1b6-d0ff4dc43a09 service neutron] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:50:51.371973 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 666/1323] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 14:50:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:50:53.325787 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3362500d-a2bd-44c2-a304-f03c30dc842f None None] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:50:53.326340 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3362500d-a2bd-44c2-a304-f03c30dc842f None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:50:53.326340 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3362500d-a2bd-44c2-a304-f03c30dc842f None None] PATH_INFO: `/` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:50:53.327698 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 658/1324] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 1 14:50:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:50:53.333302 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d2c52f-44f3-4711-b769-6126c2160868 None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:50:53.333712 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d2c52f-44f3-4711-b769-6126c2160868 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:50:53.333841 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d2c52f-44f3-4711-b769-6126c2160868 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:50:53.398982 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-73d2c52f-44f3-4711-b769-6126c2160868 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:50:53.400186 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-73d2c52f-44f3-4711-b769-6126c2160868 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:50:53.498804 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 667/1325] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 1 14:50:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:50:53.794594 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59641520-35f3-4680-9bec-069673d97dcc None None] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:50:53.794882 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59641520-35f3-4680-9bec-069673d97dcc None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:50:53.795156 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59641520-35f3-4680-9bec-069673d97dcc None None] PATH_INFO: `/` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:50:53.796487 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 659/1326] 10.222.0.33 () {54 vars in 965 bytes} [Thu Sep 1 14:50:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:50:53.802082 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f228a4-b25a-4135-a513-f86faa7f8b19 None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:50:53.802418 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f228a4-b25a-4135-a513-f86faa7f8b19 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:50:53.802692 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f228a4-b25a-4135-a513-f86faa7f8b19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:50:53.835141 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:50:53.891239 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-47f228a4-b25a-4135-a513-f86faa7f8b19 None None] MFA Rules not processed for user `2790149639aa418bbc4e4982e93eda44`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:50:53.892449 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-47f228a4-b25a-4135-a513-f86faa7f8b19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:50:53.966880 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 668/1327] 10.222.0.33 () {58 vars in 1062 bytes} [Thu Sep 1 14:50:53 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:50:53.975795 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59641520-35f3-4680-9bec-069673d97dcc None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:50:53.977357 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-59641520-35f3-4680-9bec-069673d97dcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:50:54.083421 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59641520-35f3-4680-9bec-069673d97dcc None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:50:54.155447 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89862560-7107-4728-9c7b-f62077f9e370 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2790149639aa418bbc4e4982e93eda44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:50:54.155954 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89862560-7107-4728-9c7b-f62077f9e370 service neutron] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:50:54.156231 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89862560-7107-4728-9c7b-f62077f9e370 service neutron] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:50:54.156580 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89862560-7107-4728-9c7b-f62077f9e370 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:50:54.158183 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-89862560-7107-4728-9c7b-f62077f9e370 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:50:54.175087 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:50:54.267181 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89862560-7107-4728-9c7b-f62077f9e370 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:50:54.268625 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89862560-7107-4728-9c7b-f62077f9e370 service neutron] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:50:54.286832 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 660/1328] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 14:50:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 14:50:56.459382 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83880b98-eecc-45e9-becd-715b605c7822 None None] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:50:56.459883 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83880b98-eecc-45e9-becd-715b605c7822 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:50:56.459986 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83880b98-eecc-45e9-becd-715b605c7822 None None] PATH_INFO: `/` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:50:56.461268 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 669/1329] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 1 14:50:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:50:56.467075 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c6c550-e70b-4a00-b170-b72a09def0eb None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:50:56.467510 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c6c550-e70b-4a00-b170-b72a09def0eb None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:50:56.467611 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c6c550-e70b-4a00-b170-b72a09def0eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:50:56.543258 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-a1c6c550-e70b-4a00-b170-b72a09def0eb None None] MFA Rules not processed for user `4b65298713a24cada6ba75d786076e93`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:50:56.544402 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-a1c6c550-e70b-4a00-b170-b72a09def0eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:50:56.611291 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:50:56.644179 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 661/1330] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 1 14:50:56 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:50:56.929946 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83880b98-eecc-45e9-becd-715b605c7822 None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:50:56.931515 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-83880b98-eecc-45e9-becd-715b605c7822 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:50:57.035289 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83880b98-eecc-45e9-becd-715b605c7822 None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:50:57.056351 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:50:57.112938 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8cd1a3-9402-4bad-b542-10a2dd31a78f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2790149639aa418bbc4e4982e93eda44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:50:57.113553 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8cd1a3-9402-4bad-b542-10a2dd31a78f service neutron] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:50:57.113829 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8cd1a3-9402-4bad-b542-10a2dd31a78f service neutron] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:50:57.114094 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8cd1a3-9402-4bad-b542-10a2dd31a78f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:50:57.115762 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-5e8cd1a3-9402-4bad-b542-10a2dd31a78f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:50:57.229843 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8cd1a3-9402-4bad-b542-10a2dd31a78f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:50:57.231650 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8cd1a3-9402-4bad-b542-10a2dd31a78f service neutron] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:50:57.251070 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 670/1331] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 14:50:56 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:50:59.573789 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b21e1b7-9ade-4dde-9297-ba54b7043c8c None None] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:50:59.574341 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b21e1b7-9ade-4dde-9297-ba54b7043c8c None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:50:59.574341 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b21e1b7-9ade-4dde-9297-ba54b7043c8c None None] PATH_INFO: `/` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:50:59.575455 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 662/1332] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 1 14:50:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 14:50:59.581209 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75126be-d726-4088-b109-897c031fa587 None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:50:59.581612 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75126be-d726-4088-b109-897c031fa587 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:50:59.581746 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75126be-d726-4088-b109-897c031fa587 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:50:59.645004 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:50:59.648030 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-d75126be-d726-4088-b109-897c031fa587 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:50:59.649780 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-d75126be-d726-4088-b109-897c031fa587 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:50:59.747801 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 671/1333] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 1 14:50:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:51:00.031041 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b21e1b7-9ade-4dde-9297-ba54b7043c8c None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:51:00.032588 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-4b21e1b7-9ade-4dde-9297-ba54b7043c8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:51:00.133596 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b21e1b7-9ade-4dde-9297-ba54b7043c8c None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:51:00.208783 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533b4013-3ccf-484b-ba24-4d13b6bd0720 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2790149639aa418bbc4e4982e93eda44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:51:00.209383 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533b4013-3ccf-484b-ba24-4d13b6bd0720 service neutron] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:51:00.209662 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533b4013-3ccf-484b-ba24-4d13b6bd0720 service neutron] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:51:00.209933 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533b4013-3ccf-484b-ba24-4d13b6bd0720 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:51:00.211579 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-533b4013-3ccf-484b-ba24-4d13b6bd0720 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:51:00.239868 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:51:00.320888 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533b4013-3ccf-484b-ba24-4d13b6bd0720 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:51:00.322420 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533b4013-3ccf-484b-ba24-4d13b6bd0720 service neutron] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:51:00.339664 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 663/1334] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 14:51:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:51:02.267242 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5366d34-b0e3-44cc-a2bb-7cc6c5ee27e9 None None] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:51:02.267806 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5366d34-b0e3-44cc-a2bb-7cc6c5ee27e9 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:51:02.267806 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5366d34-b0e3-44cc-a2bb-7cc6c5ee27e9 None None] PATH_INFO: `/` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:51:02.269134 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 672/1335] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 1 14:51:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:51:02.274272 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03fce16-fd17-408f-9f55-21dcb42a53ab None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:51:02.274692 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03fce16-fd17-408f-9f55-21dcb42a53ab None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:51:02.274794 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03fce16-fd17-408f-9f55-21dcb42a53ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:51:02.340062 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-a03fce16-fd17-408f-9f55-21dcb42a53ab None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:51:02.341253 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-a03fce16-fd17-408f-9f55-21dcb42a53ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:51:02.413897 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:51:02.436150 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 664/1336] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 1 14:51:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:51:02.725534 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5366d34-b0e3-44cc-a2bb-7cc6c5ee27e9 None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:51:02.728405 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-f5366d34-b0e3-44cc-a2bb-7cc6c5ee27e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:51:02.833521 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5366d34-b0e3-44cc-a2bb-7cc6c5ee27e9 None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:51:02.861819 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:51:02.906011 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9cb34f-2a5c-4590-9ab7-29a7b71adb12 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2790149639aa418bbc4e4982e93eda44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:51:02.907182 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9cb34f-2a5c-4590-9ab7-29a7b71adb12 service neutron] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:51:02.907460 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9cb34f-2a5c-4590-9ab7-29a7b71adb12 service neutron] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:51:02.907732 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9cb34f-2a5c-4590-9ab7-29a7b71adb12 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:51:02.909333 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-0c9cb34f-2a5c-4590-9ab7-29a7b71adb12 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:51:03.014666 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9cb34f-2a5c-4590-9ab7-29a7b71adb12 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:51:03.016613 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9cb34f-2a5c-4590-9ab7-29a7b71adb12 service neutron] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:51:03.035079 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 673/1337] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 14:51:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:51:05.386226 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23415975-2415-4f4b-8c1f-128353db7c8a None None] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:51:05.386805 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23415975-2415-4f4b-8c1f-128353db7c8a None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:51:05.386805 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23415975-2415-4f4b-8c1f-128353db7c8a None None] PATH_INFO: `/` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:51:05.388132 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 665/1338] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 1 14:51:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:51:05.393174 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad83cc64-a4d0-4a59-a920-f945d7f98d82 None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:51:05.393576 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad83cc64-a4d0-4a59-a920-f945d7f98d82 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:51:05.393761 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad83cc64-a4d0-4a59-a920-f945d7f98d82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:51:05.457971 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-ad83cc64-a4d0-4a59-a920-f945d7f98d82 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:51:05.459201 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-ad83cc64-a4d0-4a59-a920-f945d7f98d82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:51:05.467224 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:51:05.554716 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 674/1339] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 1 14:51:05 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:51:05.834075 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23415975-2415-4f4b-8c1f-128353db7c8a None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:51:05.835637 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-23415975-2415-4f4b-8c1f-128353db7c8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:51:05.935962 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23415975-2415-4f4b-8c1f-128353db7c8a None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:51:06.006358 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45a7582-2f08-46db-b4c0-ee9a383e2915 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2790149639aa418bbc4e4982e93eda44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:51:06.006958 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45a7582-2f08-46db-b4c0-ee9a383e2915 service neutron] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:51:06.007236 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45a7582-2f08-46db-b4c0-ee9a383e2915 service neutron] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:51:06.007507 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45a7582-2f08-46db-b4c0-ee9a383e2915 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:51:06.009079 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-f45a7582-2f08-46db-b4c0-ee9a383e2915 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:51:06.044066 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:51:06.118702 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45a7582-2f08-46db-b4c0-ee9a383e2915 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:51:06.120135 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45a7582-2f08-46db-b4c0-ee9a383e2915 service neutron] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:51:06.137977 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 666/1340] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 14:51:05 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:51:10.038010 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803459b5-c615-4afe-a1ed-af7e8e685ff9 None None] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:51:10.038621 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803459b5-c615-4afe-a1ed-af7e8e685ff9 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:51:10.038621 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803459b5-c615-4afe-a1ed-af7e8e685ff9 None None] PATH_INFO: `/` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:51:10.039924 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 675/1341] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 1 14:51:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:51:10.045055 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcb12c4-8cc2-43b9-af4d-ffc3649e6a74 None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:51:10.045479 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcb12c4-8cc2-43b9-af4d-ffc3649e6a74 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:51:10.045619 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcb12c4-8cc2-43b9-af4d-ffc3649e6a74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:51:10.114995 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-cfcb12c4-8cc2-43b9-af4d-ffc3649e6a74 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:51:10.116222 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-cfcb12c4-8cc2-43b9-af4d-ffc3649e6a74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:51:10.201258 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:51:10.214026 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 667/1342] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 1 14:51:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:51:10.499755 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803459b5-c615-4afe-a1ed-af7e8e685ff9 None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:51:10.501789 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-803459b5-c615-4afe-a1ed-af7e8e685ff9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:51:10.606185 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803459b5-c615-4afe-a1ed-af7e8e685ff9 None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:51:10.644240 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:51:10.683976 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431d8064-03ee-4c4c-b50c-c8f36aa1cba0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2790149639aa418bbc4e4982e93eda44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:51:10.684581 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431d8064-03ee-4c4c-b50c-c8f36aa1cba0 service neutron] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:51:10.684858 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431d8064-03ee-4c4c-b50c-c8f36aa1cba0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:51:10.685125 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431d8064-03ee-4c4c-b50c-c8f36aa1cba0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:51:10.686748 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-431d8064-03ee-4c4c-b50c-c8f36aa1cba0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:51:10.799327 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431d8064-03ee-4c4c-b50c-c8f36aa1cba0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:51:10.800686 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431d8064-03ee-4c4c-b50c-c8f36aa1cba0 service neutron] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:51:10.819489 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 676/1343] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 14:51:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:51:13.323684 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c02d13-0440-46a3-8ad4-f7e0ffbe723d None None] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:51:13.324239 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c02d13-0440-46a3-8ad4-f7e0ffbe723d None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:51:13.324239 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c02d13-0440-46a3-8ad4-f7e0ffbe723d None None] PATH_INFO: `/` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:51:13.325546 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 668/1344] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 1 14:51:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:51:13.331350 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd051a5-6622-4e27-b616-7771d7182c3d None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:51:13.331829 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd051a5-6622-4e27-b616-7771d7182c3d None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:51:13.331919 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd051a5-6622-4e27-b616-7771d7182c3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:51:13.397289 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-7fd051a5-6622-4e27-b616-7771d7182c3d None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:51:13.398578 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-7fd051a5-6622-4e27-b616-7771d7182c3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:51:13.416522 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:51:13.494816 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 677/1345] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 1 14:51:13 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:51:13.771721 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c02d13-0440-46a3-8ad4-f7e0ffbe723d None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:51:13.773253 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-b7c02d13-0440-46a3-8ad4-f7e0ffbe723d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:51:13.876104 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c02d13-0440-46a3-8ad4-f7e0ffbe723d None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:51:13.951592 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c862cb09-b4a8-400b-a1ef-b176130d1dbc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2790149639aa418bbc4e4982e93eda44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:51:13.952210 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c862cb09-b4a8-400b-a1ef-b176130d1dbc service neutron] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:51:13.952484 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c862cb09-b4a8-400b-a1ef-b176130d1dbc service neutron] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:51:13.952790 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c862cb09-b4a8-400b-a1ef-b176130d1dbc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:51:13.954592 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-c862cb09-b4a8-400b-a1ef-b176130d1dbc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:51:13.999490 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:51:14.072094 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c862cb09-b4a8-400b-a1ef-b176130d1dbc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:51:14.073761 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c862cb09-b4a8-400b-a1ef-b176130d1dbc service neutron] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:51:14.095276 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 669/1346] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 14:51:13 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:51:18.186268 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d7624f-729b-4985-9973-f54420df6793 None None] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:51:18.186798 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d7624f-729b-4985-9973-f54420df6793 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:51:18.186798 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d7624f-729b-4985-9973-f54420df6793 None None] PATH_INFO: `/` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:51:18.188887 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 678/1347] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 1 14:51:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:51:18.194222 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d55b04-94c6-4b62-91f5-b1b732136569 None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:51:18.194679 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d55b04-94c6-4b62-91f5-b1b732136569 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:51:18.194789 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d55b04-94c6-4b62-91f5-b1b732136569 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:51:18.263314 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-28d55b04-94c6-4b62-91f5-b1b732136569 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:51:18.264549 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-28d55b04-94c6-4b62-91f5-b1b732136569 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:51:18.362901 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:51:18.370430 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 670/1348] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 1 14:51:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:51:18.649189 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d7624f-729b-4985-9973-f54420df6793 None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:51:18.650781 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-00d7624f-729b-4985-9973-f54420df6793 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:51:18.753838 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d7624f-729b-4985-9973-f54420df6793 None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:51:18.800097 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:51:18.826364 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c0942f-9963-4130-844c-81828076a47e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2790149639aa418bbc4e4982e93eda44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:51:18.826962 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c0942f-9963-4130-844c-81828076a47e service neutron] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:51:18.827232 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c0942f-9963-4130-844c-81828076a47e service neutron] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:51:18.827998 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c0942f-9963-4130-844c-81828076a47e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:51:18.829600 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-e4c0942f-9963-4130-844c-81828076a47e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:51:18.935748 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c0942f-9963-4130-844c-81828076a47e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:51:18.937144 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c0942f-9963-4130-844c-81828076a47e service neutron] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:51:18.955899 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 679/1349] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 14:51:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:51:23.815241 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49aa6b79-1b0d-4c17-bf3e-38a150623bfe None None] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:51:23.816172 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49aa6b79-1b0d-4c17-bf3e-38a150623bfe None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:51:23.817472 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49aa6b79-1b0d-4c17-bf3e-38a150623bfe None None] PATH_INFO: `/` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:51:23.820848 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 671/1350] 10.222.0.33 () {54 vars in 921 bytes} [Thu Sep 1 14:51:23 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:51:23.826939 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b68bd1-227a-400a-9d02-36fe907203f9 None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:51:23.827413 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b68bd1-227a-400a-9d02-36fe907203f9 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:51:23.827521 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b68bd1-227a-400a-9d02-36fe907203f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:51:23.965964 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-02b68bd1-227a-400a-9d02-36fe907203f9 None None] MFA Rules not processed for user `7f312c8dcabe48ef9ea453364ade38ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:51:23.967510 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-02b68bd1-227a-400a-9d02-36fe907203f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:51:23.993864 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:51:24.085625 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 680/1351] 10.222.0.33 () {58 vars in 1018 bytes} [Thu Sep 1 14:51:23 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:51:28.417387 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49e2b35-8e7d-443a-812f-7fc680b8d5ff None None] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:51:28.417947 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49e2b35-8e7d-443a-812f-7fc680b8d5ff None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:51:28.417947 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49e2b35-8e7d-443a-812f-7fc680b8d5ff None None] PATH_INFO: `/` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:51:28.419444 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 672/1352] 10.222.0.33 () {54 vars in 921 bytes} [Thu Sep 1 14:51:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:51:28.424306 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138431c7-2895-4cfb-958a-91da1daa5ee9 None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:51:28.425279 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138431c7-2895-4cfb-958a-91da1daa5ee9 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:51:28.425786 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138431c7-2895-4cfb-958a-91da1daa5ee9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:51:28.513233 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-138431c7-2895-4cfb-958a-91da1daa5ee9 None None] MFA Rules not processed for user `7f312c8dcabe48ef9ea453364ade38ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:51:28.515555 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-138431c7-2895-4cfb-958a-91da1daa5ee9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:51:28.597786 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:51:28.638704 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 681/1353] 10.222.0.33 () {58 vars in 1018 bytes} [Thu Sep 1 14:51:28 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:51:29.822047 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d88edbb-1bf1-467b-832a-09f40cec5c04 None None] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:51:29.822614 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d88edbb-1bf1-467b-832a-09f40cec5c04 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:51:29.822794 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d88edbb-1bf1-467b-832a-09f40cec5c04 None None] PATH_INFO: `/` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:51:29.824229 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 673/1354] 10.222.0.33 () {54 vars in 921 bytes} [Thu Sep 1 14:51:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:51:29.829292 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c9b005-46e4-42a1-b0a3-9b44ad8e630d None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:51:29.829722 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c9b005-46e4-42a1-b0a3-9b44ad8e630d None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:51:29.829818 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c9b005-46e4-42a1-b0a3-9b44ad8e630d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:51:29.903914 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-f4c9b005-46e4-42a1-b0a3-9b44ad8e630d None None] MFA Rules not processed for user `7f312c8dcabe48ef9ea453364ade38ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:51:29.905122 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-f4c9b005-46e4-42a1-b0a3-9b44ad8e630d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:51:30.010670 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 682/1355] 10.222.0.33 () {58 vars in 1018 bytes} [Thu Sep 1 14:51:29 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:51:33.539296 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe10f97c-5032-45b0-92a5-bf85d55d360a None None] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:51:33.540294 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe10f97c-5032-45b0-92a5-bf85d55d360a None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:51:33.540776 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe10f97c-5032-45b0-92a5-bf85d55d360a None None] PATH_INFO: `/` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:51:33.543819 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 674/1356] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 1 14:51:33 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:51:33.550637 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc58a5f-cf05-4142-9781-dcb5e175f4f4 None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:51:33.551109 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc58a5f-cf05-4142-9781-dcb5e175f4f4 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:51:33.551109 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc58a5f-cf05-4142-9781-dcb5e175f4f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:51:33.603290 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:51:33.640594 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-4cc58a5f-cf05-4142-9781-dcb5e175f4f4 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:51:33.642758 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-4cc58a5f-cf05-4142-9781-dcb5e175f4f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:51:33.783665 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 683/1357] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 1 14:51:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:51:34.079427 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bc808c-ffa6-4f6a-a1c5-6fc2375473ff None None] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:51:34.079984 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bc808c-ffa6-4f6a-a1c5-6fc2375473ff None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:51:34.080486 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bc808c-ffa6-4f6a-a1c5-6fc2375473ff None None] PATH_INFO: `/` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:51:34.082374 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 675/1358] 10.222.0.33 () {54 vars in 962 bytes} [Thu Sep 1 14:51:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:51:34.090301 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bae6d80-420c-4f2f-a6d3-305859fe4e26 None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:51:34.090854 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bae6d80-420c-4f2f-a6d3-305859fe4e26 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:51:34.091579 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bae6d80-420c-4f2f-a6d3-305859fe4e26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:51:34.179093 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-3bae6d80-420c-4f2f-a6d3-305859fe4e26 None None] MFA Rules not processed for user `4858bb8621854ec8aba836e7d9050498`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:51:34.180688 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-3bae6d80-420c-4f2f-a6d3-305859fe4e26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:51:34.233971 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:51:34.320190 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 684/1359] 10.222.0.33 () {58 vars in 1059 bytes} [Thu Sep 1 14:51:34 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 14:51:34.328603 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bc808c-ffa6-4f6a-a1c5-6fc2375473ff None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:51:34.330207 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-06bc808c-ffa6-4f6a-a1c5-6fc2375473ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:51:34.485395 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bc808c-ffa6-4f6a-a1c5-6fc2375473ff None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:51:34.568239 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:51:34.610635 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb977ac3-a971-459b-af8a-66aa666a731b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4858bb8621854ec8aba836e7d9050498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:51:34.611302 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb977ac3-a971-459b-af8a-66aa666a731b service nova] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:51:34.611675 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb977ac3-a971-459b-af8a-66aa666a731b service nova] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:51:34.612042 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb977ac3-a971-459b-af8a-66aa666a731b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:51:34.613729 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-eb977ac3-a971-459b-af8a-66aa666a731b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:51:34.712232 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb977ac3-a971-459b-af8a-66aa666a731b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:51:34.714060 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb977ac3-a971-459b-af8a-66aa666a731b service nova] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:51:34.734133 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 676/1360] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 14:51:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:51:36.889332 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621b262c-30f2-42c9-9489-f7593e76a251 None None] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:51:36.889895 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621b262c-30f2-42c9-9489-f7593e76a251 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:51:36.889994 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621b262c-30f2-42c9-9489-f7593e76a251 None None] PATH_INFO: `/` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:51:36.894282 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 685/1361] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 1 14:51:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:51:36.896584 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a653344-678a-44b1-ad37-cf063fa185fb None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:51:36.897015 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a653344-678a-44b1-ad37-cf063fa185fb None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:51:36.897114 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a653344-678a-44b1-ad37-cf063fa185fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:51:36.966962 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-7a653344-678a-44b1-ad37-cf063fa185fb None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:51:36.968675 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-7a653344-678a-44b1-ad37-cf063fa185fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:51:37.011569 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:51:37.081525 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 677/1362] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 1 14:51:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:51:37.372124 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621b262c-30f2-42c9-9489-f7593e76a251 None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:51:37.373633 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-621b262c-30f2-42c9-9489-f7593e76a251 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:51:37.531758 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621b262c-30f2-42c9-9489-f7593e76a251 None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:51:37.541576 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:51:37.656614 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4253acca-609a-45f3-9e70-c91358b88b0a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4858bb8621854ec8aba836e7d9050498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:51:37.657803 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4253acca-609a-45f3-9e70-c91358b88b0a service nova] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:51:37.661461 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4253acca-609a-45f3-9e70-c91358b88b0a service nova] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:51:37.661461 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4253acca-609a-45f3-9e70-c91358b88b0a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:51:37.661461 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-4253acca-609a-45f3-9e70-c91358b88b0a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:51:37.766097 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4253acca-609a-45f3-9e70-c91358b88b0a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:51:37.767935 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4253acca-609a-45f3-9e70-c91358b88b0a service nova] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:51:37.790582 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 686/1363] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 14:51:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:51:40.029258 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac916b98-7d71-4e3b-853f-2d28bd2a729e None None] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:51:40.029828 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac916b98-7d71-4e3b-853f-2d28bd2a729e None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:51:40.029828 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac916b98-7d71-4e3b-853f-2d28bd2a729e None None] PATH_INFO: `/` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:51:40.031225 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 678/1364] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 1 14:51:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 14:51:40.036571 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c718d834-648e-40cd-a276-27fd165b37c7 None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:51:40.037018 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c718d834-648e-40cd-a276-27fd165b37c7 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:51:40.037140 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c718d834-648e-40cd-a276-27fd165b37c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:51:40.055132 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:51:40.110614 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-c718d834-648e-40cd-a276-27fd165b37c7 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:51:40.111851 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-c718d834-648e-40cd-a276-27fd165b37c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:51:40.214158 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 687/1365] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 1 14:51:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:51:40.506628 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a17266-258c-4c79-af0a-94f9fb55784f None None] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:51:40.507131 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a17266-258c-4c79-af0a-94f9fb55784f None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:51:40.507503 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a17266-258c-4c79-af0a-94f9fb55784f None None] PATH_INFO: `/` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:51:40.509075 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 679/1366] 10.222.0.33 () {54 vars in 962 bytes} [Thu Sep 1 14:51:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:51:40.514384 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff16a58e-337c-48a3-a3c5-3691f2e6cf97 None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:51:40.514680 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff16a58e-337c-48a3-a3c5-3691f2e6cf97 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:51:40.514972 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff16a58e-337c-48a3-a3c5-3691f2e6cf97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:51:40.593944 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-ff16a58e-337c-48a3-a3c5-3691f2e6cf97 None None] MFA Rules not processed for user `4858bb8621854ec8aba836e7d9050498`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:51:40.595196 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-ff16a58e-337c-48a3-a3c5-3691f2e6cf97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:51:40.632854 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:51:40.725969 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 688/1367] 10.222.0.33 () {58 vars in 1059 bytes} [Thu Sep 1 14:51:40 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:51:40.734454 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a17266-258c-4c79-af0a-94f9fb55784f None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:51:40.735962 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-55a17266-258c-4c79-af0a-94f9fb55784f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:51:40.904844 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a17266-258c-4c79-af0a-94f9fb55784f None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:51:40.927067 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:51:41.027273 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedc67ed-e2ae-4a39-88e3-cebd502aac7a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4858bb8621854ec8aba836e7d9050498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:51:41.028166 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedc67ed-e2ae-4a39-88e3-cebd502aac7a service nova] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:51:41.028448 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedc67ed-e2ae-4a39-88e3-cebd502aac7a service nova] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:51:41.028720 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedc67ed-e2ae-4a39-88e3-cebd502aac7a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:51:41.030867 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-eedc67ed-e2ae-4a39-88e3-cebd502aac7a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:51:41.128592 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eedc67ed-e2ae-4a39-88e3-cebd502aac7a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:51:41.130433 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eedc67ed-e2ae-4a39-88e3-cebd502aac7a service nova] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:51:41.149805 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 680/1368] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 14:51:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:51:41.725382 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce19ffa-d191-40d3-8479-abf0a3a23bbc None None] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:51:41.725911 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce19ffa-d191-40d3-8479-abf0a3a23bbc None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:51:41.725911 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce19ffa-d191-40d3-8479-abf0a3a23bbc None None] PATH_INFO: `/` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:51:41.727462 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 689/1369] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 1 14:51:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:51:41.732457 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11591623-b3fc-45c2-b83d-ac76f373f1ce None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:51:41.732748 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11591623-b3fc-45c2-b83d-ac76f373f1ce None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:51:41.733024 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11591623-b3fc-45c2-b83d-ac76f373f1ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:51:41.766471 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:51:41.803791 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-11591623-b3fc-45c2-b83d-ac76f373f1ce None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:51:41.804935 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-11591623-b3fc-45c2-b83d-ac76f373f1ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:51:41.907156 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 681/1370] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 1 14:51:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:51:42.494183 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9912a76-abc0-4a48-8918-c852d5dd1ee7 None None] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:51:42.494183 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9912a76-abc0-4a48-8918-c852d5dd1ee7 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:51:42.494183 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9912a76-abc0-4a48-8918-c852d5dd1ee7 None None] PATH_INFO: `/` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:51:42.494183 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 690/1371] 10.222.0.33 () {54 vars in 964 bytes} [Thu Sep 1 14:51:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:51:42.498139 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462ec3c1-f169-4010-8af6-906a12242912 None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:51:42.499476 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462ec3c1-f169-4010-8af6-906a12242912 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:51:42.499476 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462ec3c1-f169-4010-8af6-906a12242912 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:51:42.579234 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-462ec3c1-f169-4010-8af6-906a12242912 None None] MFA Rules not processed for user `d3b81739e34e4e648f88b6d73fcb5e46`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:51:42.580457 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-462ec3c1-f169-4010-8af6-906a12242912 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:51:42.629682 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:51:42.664839 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 682/1372] 10.222.0.33 () {58 vars in 1061 bytes} [Thu Sep 1 14:51:42 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:51:42.668529 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9912a76-abc0-4a48-8918-c852d5dd1ee7 None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:51:42.670051 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-b9912a76-abc0-4a48-8918-c852d5dd1ee7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:51:42.785943 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9912a76-abc0-4a48-8918-c852d5dd1ee7 None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:51:42.831416 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:51:42.874322 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe04b62-e439-4ec6-856f-ef8dccd69b2c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3b81739e34e4e648f88b6d73fcb5e46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:51:42.874322 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe04b62-e439-4ec6-856f-ef8dccd69b2c service cinder] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:51:42.874322 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe04b62-e439-4ec6-856f-ef8dccd69b2c service cinder] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:51:42.874322 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe04b62-e439-4ec6-856f-ef8dccd69b2c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:51:42.876742 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-8fe04b62-e439-4ec6-856f-ef8dccd69b2c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:51:42.992110 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe04b62-e439-4ec6-856f-ef8dccd69b2c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:51:42.993587 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe04b62-e439-4ec6-856f-ef8dccd69b2c service cinder] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:51:43.013074 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 691/1373] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 14:51:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:51:43.145974 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3339aa6b-20e1-43d7-aa91-8dbc0cd44c70 None None] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:51:43.146227 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3339aa6b-20e1-43d7-aa91-8dbc0cd44c70 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:51:43.146729 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3339aa6b-20e1-43d7-aa91-8dbc0cd44c70 None None] PATH_INFO: `/` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:51:43.148020 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 683/1374] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 1 14:51:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:51:43.153305 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fd1bbe-6272-46b0-8392-667c497e879a None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:51:43.153558 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fd1bbe-6272-46b0-8392-667c497e879a None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:51:43.153834 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fd1bbe-6272-46b0-8392-667c497e879a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:51:43.245695 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-a5fd1bbe-6272-46b0-8392-667c497e879a None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:51:43.247647 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-a5fd1bbe-6272-46b0-8392-667c497e879a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:51:43.272686 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:51:43.359426 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 692/1375] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 1 14:51:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:51:43.645752 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3339aa6b-20e1-43d7-aa91-8dbc0cd44c70 None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:51:43.647312 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-3339aa6b-20e1-43d7-aa91-8dbc0cd44c70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:51:43.799279 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3339aa6b-20e1-43d7-aa91-8dbc0cd44c70 None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:51:43.850791 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:51:43.912733 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9899205-de31-40eb-ba34-0faffd6f4ba7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4858bb8621854ec8aba836e7d9050498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:51:43.913338 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9899205-de31-40eb-ba34-0faffd6f4ba7 service nova] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:51:43.913621 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9899205-de31-40eb-ba34-0faffd6f4ba7 service nova] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:51:43.913909 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9899205-de31-40eb-ba34-0faffd6f4ba7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:51:43.915538 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-a9899205-de31-40eb-ba34-0faffd6f4ba7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:51:44.004511 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9899205-de31-40eb-ba34-0faffd6f4ba7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:51:44.006136 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9899205-de31-40eb-ba34-0faffd6f4ba7 service nova] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:51:44.024333 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 684/1376] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 14:51:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:51:45.954030 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a32d5a-9640-49ab-ad5a-e9b83de249bb None None] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:51:45.954586 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a32d5a-9640-49ab-ad5a-e9b83de249bb None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:51:45.954586 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a32d5a-9640-49ab-ad5a-e9b83de249bb None None] PATH_INFO: `/` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:51:45.955953 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 693/1377] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 1 14:51:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:51:45.961124 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3117e06c-f1b5-44e2-94e2-a30b28f3d833 None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:51:45.961526 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3117e06c-f1b5-44e2-94e2-a30b28f3d833 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:51:45.961680 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3117e06c-f1b5-44e2-94e2-a30b28f3d833 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:51:46.034922 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-3117e06c-f1b5-44e2-94e2-a30b28f3d833 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:51:46.036206 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-3117e06c-f1b5-44e2-94e2-a30b28f3d833 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:51:46.044997 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:51:46.153843 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 685/1378] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 1 14:51:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:51:46.443439 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a32d5a-9640-49ab-ad5a-e9b83de249bb None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:51:46.444951 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-e8a32d5a-9640-49ab-ad5a-e9b83de249bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:51:46.636226 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a32d5a-9640-49ab-ad5a-e9b83de249bb None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:51:46.640765 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:51:46.763768 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f393b6fe-56bf-4958-a6a8-5d1ce41f644a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4858bb8621854ec8aba836e7d9050498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:51:46.764834 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f393b6fe-56bf-4958-a6a8-5d1ce41f644a service nova] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:51:46.765113 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f393b6fe-56bf-4958-a6a8-5d1ce41f644a service nova] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:51:46.765375 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f393b6fe-56bf-4958-a6a8-5d1ce41f644a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:51:46.767088 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-f393b6fe-56bf-4958-a6a8-5d1ce41f644a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:51:46.864050 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f393b6fe-56bf-4958-a6a8-5d1ce41f644a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:51:46.865713 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f393b6fe-56bf-4958-a6a8-5d1ce41f644a service nova] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:51:46.884912 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 694/1379] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 14:51:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 442 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:51:48.832022 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec36647-e3ef-49a4-9b2c-d63b0dbb1856 None None] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:51:48.832568 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec36647-e3ef-49a4-9b2c-d63b0dbb1856 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:51:48.832568 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec36647-e3ef-49a4-9b2c-d63b0dbb1856 None None] PATH_INFO: `/` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:51:48.834053 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 686/1380] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 1 14:51:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:51:48.839063 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0340937f-92fe-4ffd-b248-4210506b1191 None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:51:48.839450 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0340937f-92fe-4ffd-b248-4210506b1191 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:51:48.839757 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0340937f-92fe-4ffd-b248-4210506b1191 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:51:48.851552 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:51:48.909299 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-0340937f-92fe-4ffd-b248-4210506b1191 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:51:48.910562 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-0340937f-92fe-4ffd-b248-4210506b1191 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:51:49.010093 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 695/1381] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 1 14:51:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:51:49.295303 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec36647-e3ef-49a4-9b2c-d63b0dbb1856 None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:51:49.297654 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-cec36647-e3ef-49a4-9b2c-d63b0dbb1856 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:51:49.451253 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:51:49.464277 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec36647-e3ef-49a4-9b2c-d63b0dbb1856 None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:51:49.590944 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ab91be-377d-432f-843c-b34bc0afd816 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4858bb8621854ec8aba836e7d9050498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:51:49.591559 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ab91be-377d-432f-843c-b34bc0afd816 service nova] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:51:49.591839 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ab91be-377d-432f-843c-b34bc0afd816 service nova] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:51:49.592846 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ab91be-377d-432f-843c-b34bc0afd816 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:51:49.594617 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-01ab91be-377d-432f-843c-b34bc0afd816 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:51:49.697673 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ab91be-377d-432f-843c-b34bc0afd816 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:51:49.699436 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ab91be-377d-432f-843c-b34bc0afd816 service nova] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:51:49.711314 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:51:49.721515 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 687/1382] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 14:51:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:51:51.637343 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04346b7c-36a6-4257-a0af-2fea583fae78 None None] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:51:51.637890 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04346b7c-36a6-4257-a0af-2fea583fae78 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:51:51.637890 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04346b7c-36a6-4257-a0af-2fea583fae78 None None] PATH_INFO: `/` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:51:51.639319 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 696/1383] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 1 14:51:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:51:51.644526 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a43c68-e518-4644-8dd6-cb63bdf09cac None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:51:51.645097 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a43c68-e518-4644-8dd6-cb63bdf09cac None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:51:51.645097 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a43c68-e518-4644-8dd6-cb63bdf09cac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:51:51.711731 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-15a43c68-e518-4644-8dd6-cb63bdf09cac None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:51:51.712941 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-15a43c68-e518-4644-8dd6-cb63bdf09cac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:51:51.838902 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 688/1384] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 1 14:51:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:51:52.128292 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04346b7c-36a6-4257-a0af-2fea583fae78 None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:51:52.129838 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-04346b7c-36a6-4257-a0af-2fea583fae78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:51:52.215121 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:51:52.281652 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04346b7c-36a6-4257-a0af-2fea583fae78 None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:51:52.399350 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bc78a8-ee31-46e7-b325-864d1f349a3c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4858bb8621854ec8aba836e7d9050498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:51:52.399882 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bc78a8-ee31-46e7-b325-864d1f349a3c service nova] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:51:52.400160 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bc78a8-ee31-46e7-b325-864d1f349a3c service nova] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:51:52.400436 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bc78a8-ee31-46e7-b325-864d1f349a3c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:51:52.402052 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-33bc78a8-ee31-46e7-b325-864d1f349a3c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:51:52.454269 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:51:52.497757 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bc78a8-ee31-46e7-b325-864d1f349a3c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:51:52.500616 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bc78a8-ee31-46e7-b325-864d1f349a3c service nova] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:51:52.522605 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 697/1385] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 14:51:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:51:54.405198 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfe61dd-28ad-4872-a737-6db575e175d0 None None] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:51:54.405744 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfe61dd-28ad-4872-a737-6db575e175d0 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:51:54.406043 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfe61dd-28ad-4872-a737-6db575e175d0 None None] PATH_INFO: `/` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:51:54.407880 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 689/1386] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 1 14:51:54 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:51:54.413431 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb83fc4-6088-4cf2-bfdb-f6d58a87812d None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:51:54.414123 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb83fc4-6088-4cf2-bfdb-f6d58a87812d None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:51:54.414512 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb83fc4-6088-4cf2-bfdb-f6d58a87812d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:51:54.496963 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-8cb83fc4-6088-4cf2-bfdb-f6d58a87812d None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:51:54.498194 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-8cb83fc4-6088-4cf2-bfdb-f6d58a87812d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:51:54.596243 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 698/1387] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 1 14:51:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:51:54.976125 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfe61dd-28ad-4872-a737-6db575e175d0 None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:51:54.978208 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-abfe61dd-28ad-4872-a737-6db575e175d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:51:55.071276 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:51:55.145941 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfe61dd-28ad-4872-a737-6db575e175d0 None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:51:55.260956 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac22271-ab3a-4d7a-842a-899ae2dee548 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4858bb8621854ec8aba836e7d9050498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:51:55.261726 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac22271-ab3a-4d7a-842a-899ae2dee548 service nova] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:51:55.262085 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac22271-ab3a-4d7a-842a-899ae2dee548 service nova] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:51:55.262465 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac22271-ab3a-4d7a-842a-899ae2dee548 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:51:55.264782 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-bac22271-ab3a-4d7a-842a-899ae2dee548 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:51:55.306754 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:51:55.361059 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac22271-ab3a-4d7a-842a-899ae2dee548 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:51:55.363651 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac22271-ab3a-4d7a-842a-899ae2dee548 service nova] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:51:55.396936 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 690/1388] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 14:51:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:51:57.468077 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7365388-75b4-4df1-81e8-b6db801d78df None None] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:51:57.468615 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7365388-75b4-4df1-81e8-b6db801d78df None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:51:57.468615 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7365388-75b4-4df1-81e8-b6db801d78df None None] PATH_INFO: `/` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:51:57.469966 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 699/1389] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 1 14:51:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:51:57.475209 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe706d7-334c-49bd-b2e6-ec4379f8f4f1 None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:51:57.475626 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe706d7-334c-49bd-b2e6-ec4379f8f4f1 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:51:57.475866 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe706d7-334c-49bd-b2e6-ec4379f8f4f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:51:57.544179 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-3fe706d7-334c-49bd-b2e6-ec4379f8f4f1 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:51:57.546008 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-3fe706d7-334c-49bd-b2e6-ec4379f8f4f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:51:57.643284 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 691/1390] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 1 14:51:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:51:57.932703 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7365388-75b4-4df1-81e8-b6db801d78df None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:51:57.934279 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-d7365388-75b4-4df1-81e8-b6db801d78df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:51:57.956927 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc9f635-9728-48f6-b97a-3a6ec3afaa40 None None] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:51:57.957209 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc9f635-9728-48f6-b97a-3a6ec3afaa40 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:51:57.957510 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc9f635-9728-48f6-b97a-3a6ec3afaa40 None None] PATH_INFO: `/` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:51:57.959051 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 692/1391] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 1 14:51:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:51:57.963799 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66735a6d-ec18-4694-bde9-c45b5c8b6413 None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:51:57.964088 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66735a6d-ec18-4694-bde9-c45b5c8b6413 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:51:57.964365 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66735a6d-ec18-4694-bde9-c45b5c8b6413 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:51:57.966840 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:51:57.981260 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:51:58.034274 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-66735a6d-ec18-4694-bde9-c45b5c8b6413 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:51:58.035491 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-66735a6d-ec18-4694-bde9-c45b5c8b6413 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:51:58.085338 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7365388-75b4-4df1-81e8-b6db801d78df None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:51:58.134103 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 693/1392] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 1 14:51:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 14:51:58.185236 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:51:58.198551 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a0bb55-0739-4d5e-9daa-038cc851e51f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4858bb8621854ec8aba836e7d9050498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:51:58.199160 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a0bb55-0739-4d5e-9daa-038cc851e51f service nova] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:51:58.199441 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a0bb55-0739-4d5e-9daa-038cc851e51f service nova] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:51:58.199707 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a0bb55-0739-4d5e-9daa-038cc851e51f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:51:58.201325 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-b9a0bb55-0739-4d5e-9daa-038cc851e51f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:51:58.289169 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a0bb55-0739-4d5e-9daa-038cc851e51f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:51:58.290833 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a0bb55-0739-4d5e-9daa-038cc851e51f service nova] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:51:58.308474 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 700/1393] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 14:51:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 14:51:58.443207 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249399c2-8d70-45f4-bc2e-dcb01967d82d None None] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:51:58.443491 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249399c2-8d70-45f4-bc2e-dcb01967d82d None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:51:58.443854 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249399c2-8d70-45f4-bc2e-dcb01967d82d None None] PATH_INFO: `/` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:51:58.445261 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 694/1394] 10.222.0.33 () {54 vars in 967 bytes} [Thu Sep 1 14:51:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:51:58.450780 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb917dd7-ca0f-43da-bd51-46ab37e53e7e None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:51:58.451151 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb917dd7-ca0f-43da-bd51-46ab37e53e7e None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:51:58.451493 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb917dd7-ca0f-43da-bd51-46ab37e53e7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:51:58.534637 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-bb917dd7-ca0f-43da-bd51-46ab37e53e7e None None] MFA Rules not processed for user `3069f72ce98049b99c9d98229a62bb51`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:51:58.535861 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-bb917dd7-ca0f-43da-bd51-46ab37e53e7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:51:58.588830 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:51:58.613119 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 701/1395] 10.222.0.33 () {58 vars in 1064 bytes} [Thu Sep 1 14:51:58 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:51:58.620316 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249399c2-8d70-45f4-bc2e-dcb01967d82d None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:51:58.621839 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-249399c2-8d70-45f4-bc2e-dcb01967d82d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:51:58.721721 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:51:58.730100 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249399c2-8d70-45f4-bc2e-dcb01967d82d None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:51:58.803765 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690d2399-1a1c-4d71-8113-9b0cb8cc1c81 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3069f72ce98049b99c9d98229a62bb51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:51:58.804365 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690d2399-1a1c-4d71-8113-9b0cb8cc1c81 service glance] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:51:58.804639 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690d2399-1a1c-4d71-8113-9b0cb8cc1c81 service glance] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:51:58.804905 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690d2399-1a1c-4d71-8113-9b0cb8cc1c81 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:51:58.806514 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-690d2399-1a1c-4d71-8113-9b0cb8cc1c81 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:51:58.915184 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690d2399-1a1c-4d71-8113-9b0cb8cc1c81 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:51:58.916644 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690d2399-1a1c-4d71-8113-9b0cb8cc1c81 service glance] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:51:58.934802 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 695/1396] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 14:51:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:51:59.678463 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe156ba-6214-4b45-890a-20e487bf08be None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:51:59.678945 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe156ba-6214-4b45-890a-20e487bf08be None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:51:59.679254 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe156ba-6214-4b45-890a-20e487bf08be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:51:59.746251 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-dfe156ba-6214-4b45-890a-20e487bf08be None None] MFA Rules not processed for user `24b6f275ffb04ffeae3c2097bfd040f5`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:51:59.747557 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-dfe156ba-6214-4b45-890a-20e487bf08be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:51:59.835525 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 702/1397] 10.222.0.33 () {58 vars in 1009 bytes} [Thu Sep 1 14:51:59 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:51:59.878574 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545d2b26-ffa5-4cfa-a5e3-5182cce9dc48 None None] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:51:59.879065 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545d2b26-ffa5-4cfa-a5e3-5182cce9dc48 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:51:59.879241 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545d2b26-ffa5-4cfa-a5e3-5182cce9dc48 None None] PATH_INFO: `/` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:51:59.880696 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 696/1398] 10.222.0.33 () {54 vars in 943 bytes} [Thu Sep 1 14:51:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:51:59.887255 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4161826-26a0-47d1-bea4-d19bfbcd0b28 None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:51:59.887546 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4161826-26a0-47d1-bea4-d19bfbcd0b28 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:51:59.887823 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4161826-26a0-47d1-bea4-d19bfbcd0b28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:51:59.964498 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-e4161826-26a0-47d1-bea4-d19bfbcd0b28 None None] MFA Rules not processed for user `73a5bda5a6ea46e9863ccafd57dee339`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:51:59.965587 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-e4161826-26a0-47d1-bea4-d19bfbcd0b28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:51:59.978168 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:52:00.074415 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 703/1399] 10.222.0.33 () {58 vars in 1040 bytes} [Thu Sep 1 14:51:59 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:52:00.083889 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545d2b26-ffa5-4cfa-a5e3-5182cce9dc48 None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:52:00.085413 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-545d2b26-ffa5-4cfa-a5e3-5182cce9dc48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:52:00.109750 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:52:00.219316 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545d2b26-ffa5-4cfa-a5e3-5182cce9dc48 None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:52:00.251451 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87619844-25e5-4ecd-ab0a-449337256fdd None None] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:52:00.252154 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87619844-25e5-4ecd-ab0a-449337256fdd None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:52:00.252443 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87619844-25e5-4ecd-ab0a-449337256fdd None None] PATH_INFO: `/` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:52:00.254281 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 704/1400] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 1 14:52:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 14:52:00.259049 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b44c712-0915-446c-93c6-96b73d12e887 None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:52:00.259336 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b44c712-0915-446c-93c6-96b73d12e887 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:52:00.259640 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b44c712-0915-446c-93c6-96b73d12e887 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:52:00.323029 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8c8061-bb37-4c13-8b3c-d711a0b3bb71 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73a5bda5a6ea46e9863ccafd57dee339', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:52:00.323633 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8c8061-bb37-4c13-8b3c-d711a0b3bb71 service swift] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:52:00.323910 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8c8061-bb37-4c13-8b3c-d711a0b3bb71 service swift] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:52:00.324176 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8c8061-bb37-4c13-8b3c-d711a0b3bb71 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:52:00.325811 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-5f8c8061-bb37-4c13-8b3c-d711a0b3bb71 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:52:00.328010 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-9b44c712-0915-446c-93c6-96b73d12e887 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:52:00.329237 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-9b44c712-0915-446c-93c6-96b73d12e887 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:52:00.333493 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:52:00.405861 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:52:00.410107 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8c8061-bb37-4c13-8b3c-d711a0b3bb71 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:52:00.411770 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8c8061-bb37-4c13-8b3c-d711a0b3bb71 service swift] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:52:00.419810 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 697/1401] 10.222.0.33 () {60 vars in 1384 bytes} [Thu Sep 1 14:52:00 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 01 14:52:00.429122 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 705/1402] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 1 14:52:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 14:52:00.719334 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8c8061-bb37-4c13-8b3c-d711a0b3bb71 service swift] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:52:00.720561 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-5f8c8061-bb37-4c13-8b3c-d711a0b3bb71 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:52:00.826737 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8c8061-bb37-4c13-8b3c-d711a0b3bb71 service swift] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:52:00.887328 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:52:00.960912 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b141741b-450b-459f-9d5e-44b71625cecd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4858bb8621854ec8aba836e7d9050498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:52:00.961689 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b141741b-450b-459f-9d5e-44b71625cecd service nova] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:52:00.962030 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b141741b-450b-459f-9d5e-44b71625cecd service nova] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:52:00.962398 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b141741b-450b-459f-9d5e-44b71625cecd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:52:00.964088 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-b141741b-450b-459f-9d5e-44b71625cecd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:52:01.063471 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b141741b-450b-459f-9d5e-44b71625cecd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:52:01.065296 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b141741b-450b-459f-9d5e-44b71625cecd service nova] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:52:01.088183 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 698/1403] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 14:52:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:52:03.044810 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffbace9-9e3f-4f5a-a6c5-d1e89a8933fe None None] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:52:03.045906 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffbace9-9e3f-4f5a-a6c5-d1e89a8933fe None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:52:03.045906 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffbace9-9e3f-4f5a-a6c5-d1e89a8933fe None None] PATH_INFO: `/` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:52:03.047341 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 706/1404] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 1 14:52:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:52:03.052554 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cfe9a7-cf4c-4346-9aa2-21a2ba5dab93 None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:52:03.053053 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cfe9a7-cf4c-4346-9aa2-21a2ba5dab93 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:52:03.053151 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cfe9a7-cf4c-4346-9aa2-21a2ba5dab93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:52:03.124701 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-02cfe9a7-cf4c-4346-9aa2-21a2ba5dab93 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:52:03.125942 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-02cfe9a7-cf4c-4346-9aa2-21a2ba5dab93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:52:03.140413 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:52:03.232367 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 699/1405] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 1 14:52:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:52:03.521807 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffbace9-9e3f-4f5a-a6c5-d1e89a8933fe None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:52:03.523442 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-7ffbace9-9e3f-4f5a-a6c5-d1e89a8933fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:52:03.686767 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffbace9-9e3f-4f5a-a6c5-d1e89a8933fe None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:52:03.747383 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:52:03.921916 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eba03ec-fa7b-48e0-bc5c-3c9f819448cf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4858bb8621854ec8aba836e7d9050498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:52:03.922550 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eba03ec-fa7b-48e0-bc5c-3c9f819448cf service nova] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:52:03.922824 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eba03ec-fa7b-48e0-bc5c-3c9f819448cf service nova] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:52:03.923083 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eba03ec-fa7b-48e0-bc5c-3c9f819448cf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:52:03.924672 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-5eba03ec-fa7b-48e0-bc5c-3c9f819448cf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:52:04.021422 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eba03ec-fa7b-48e0-bc5c-3c9f819448cf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:52:04.023042 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eba03ec-fa7b-48e0-bc5c-3c9f819448cf service nova] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:52:04.041754 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 707/1406] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 14:52:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 520 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 14:52:05.991875 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d930e1d0-62d3-47f8-a3d3-8ea3295ceefb None None] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:52:05.992397 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d930e1d0-62d3-47f8-a3d3-8ea3295ceefb None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:52:05.992397 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d930e1d0-62d3-47f8-a3d3-8ea3295ceefb None None] PATH_INFO: `/` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:52:05.993679 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 700/1407] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 1 14:52:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:52:05.999582 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3a2c18-6628-4e3a-9882-f53103737870 None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:52:06.000384 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3a2c18-6628-4e3a-9882-f53103737870 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:52:06.000734 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3a2c18-6628-4e3a-9882-f53103737870 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:52:06.075921 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-1b3a2c18-6628-4e3a-9882-f53103737870 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:52:06.077262 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-1b3a2c18-6628-4e3a-9882-f53103737870 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:52:06.096209 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:52:06.192163 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 708/1408] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 1 14:52:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 14:52:06.483657 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d930e1d0-62d3-47f8-a3d3-8ea3295ceefb None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:52:06.485195 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-d930e1d0-62d3-47f8-a3d3-8ea3295ceefb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:52:06.652435 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:52:06.664250 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d930e1d0-62d3-47f8-a3d3-8ea3295ceefb None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:52:06.790337 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1f8371-3838-4591-97a0-205102b9ef72 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4858bb8621854ec8aba836e7d9050498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:52:06.790964 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1f8371-3838-4591-97a0-205102b9ef72 service nova] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:52:06.791529 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1f8371-3838-4591-97a0-205102b9ef72 service nova] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:52:06.791871 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1f8371-3838-4591-97a0-205102b9ef72 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:52:06.793921 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-7d1f8371-3838-4591-97a0-205102b9ef72 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:52:06.903617 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1f8371-3838-4591-97a0-205102b9ef72 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:52:06.905831 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1f8371-3838-4591-97a0-205102b9ef72 service nova] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:52:06.925854 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 701/1409] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 14:52:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 443 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:52:16.189794 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=35 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:52:16.923657 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=99 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:52:22.720681 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b81b656-bd51-4f29-a8bd-6ff4e596cd16 None None] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:52:22.721245 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b81b656-bd51-4f29-a8bd-6ff4e596cd16 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:52:22.721245 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b81b656-bd51-4f29-a8bd-6ff4e596cd16 None None] PATH_INFO: `/` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:52:22.722598 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 709/1410] 10.222.0.33 () {54 vars in 918 bytes} [Thu Sep 1 14:52:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:52:22.727531 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d3a579-b23c-4bd5-8cf5-506655b6c830 None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:52:22.727917 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d3a579-b23c-4bd5-8cf5-506655b6c830 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:52:22.728080 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d3a579-b23c-4bd5-8cf5-506655b6c830 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:52:22.802404 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-27d3a579-b23c-4bd5-8cf5-506655b6c830 None None] MFA Rules not processed for user `7f312c8dcabe48ef9ea453364ade38ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:52:22.803608 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-27d3a579-b23c-4bd5-8cf5-506655b6c830 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:52:22.907098 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 702/1411] 10.222.0.33 () {58 vars in 1015 bytes} [Thu Sep 1 14:52:22 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:52:22.922828 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb133fc4-6052-4906-b4a2-e61aa89edc3d None None] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:52:22.923113 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb133fc4-6052-4906-b4a2-e61aa89edc3d None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:52:22.923386 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb133fc4-6052-4906-b4a2-e61aa89edc3d None None] PATH_INFO: `/` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:52:22.924749 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 710/1412] 10.222.0.33 () {54 vars in 918 bytes} [Thu Sep 1 14:52:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:52:22.929217 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dade7b6a-a6cc-40b5-bad3-85ab3c194bbe None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:52:22.929500 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dade7b6a-a6cc-40b5-bad3-85ab3c194bbe None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:52:22.929771 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dade7b6a-a6cc-40b5-bad3-85ab3c194bbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:52:22.999918 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:52:23.002875 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-dade7b6a-a6cc-40b5-bad3-85ab3c194bbe None None] MFA Rules not processed for user `4858bb8621854ec8aba836e7d9050498`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:52:23.004726 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-dade7b6a-a6cc-40b5-bad3-85ab3c194bbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:52:23.124256 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 703/1413] 10.222.0.33 () {58 vars in 1015 bytes} [Thu Sep 1 14:52:22 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 14:52:25.488689 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2648ca63-b779-4f9d-be8d-51d37f3cc47e None None] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:52:25.489145 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2648ca63-b779-4f9d-be8d-51d37f3cc47e None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:52:25.489273 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2648ca63-b779-4f9d-be8d-51d37f3cc47e None None] PATH_INFO: `/` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:52:25.490587 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 711/1414] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 14:52:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:52:25.495959 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6187eb5c-5c1b-40bb-8736-6d3dcfbe0047 None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:52:25.496383 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6187eb5c-5c1b-40bb-8736-6d3dcfbe0047 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:52:25.496530 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6187eb5c-5c1b-40bb-8736-6d3dcfbe0047 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:52:25.562231 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-6187eb5c-5c1b-40bb-8736-6d3dcfbe0047 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:52:25.563480 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-6187eb5c-5c1b-40bb-8736-6d3dcfbe0047 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:52:25.617909 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:52:25.659950 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 704/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 14:52:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:52:25.944770 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2648ca63-b779-4f9d-be8d-51d37f3cc47e None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:52:25.946292 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-2648ca63-b779-4f9d-be8d-51d37f3cc47e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:52:26.089969 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2648ca63-b779-4f9d-be8d-51d37f3cc47e None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:52:26.160138 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:52:26.200494 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74cdea6-5dce-40d2-8e32-43e9f7940b6d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4858bb8621854ec8aba836e7d9050498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:52:26.201018 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74cdea6-5dce-40d2-8e32-43e9f7940b6d service nova] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:52:26.201293 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74cdea6-5dce-40d2-8e32-43e9f7940b6d service nova] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:52:26.201557 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74cdea6-5dce-40d2-8e32-43e9f7940b6d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:52:26.203185 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-a74cdea6-5dce-40d2-8e32-43e9f7940b6d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:52:26.289340 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74cdea6-5dce-40d2-8e32-43e9f7940b6d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:52:26.290961 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74cdea6-5dce-40d2-8e32-43e9f7940b6d service nova] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:52:26.309160 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 712/1416] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 14:52:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:52:27.192048 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e411e0c5-cbd8-4a5a-95fc-c0739e2064b2 None None] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:52:27.192590 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e411e0c5-cbd8-4a5a-95fc-c0739e2064b2 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:52:27.192590 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e411e0c5-cbd8-4a5a-95fc-c0739e2064b2 None None] PATH_INFO: `/` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:52:27.193920 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 705/1417] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 1 14:52:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:52:27.198731 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3508e126-53b3-4cd2-bc2d-f0887d82ba9a None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:52:27.199169 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3508e126-53b3-4cd2-bc2d-f0887d82ba9a None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:52:27.199275 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3508e126-53b3-4cd2-bc2d-f0887d82ba9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:52:27.273410 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-3508e126-53b3-4cd2-bc2d-f0887d82ba9a None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:52:27.274628 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-3508e126-53b3-4cd2-bc2d-f0887d82ba9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:52:27.300751 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:52:27.369025 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 713/1418] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 1 14:52:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:52:27.519166 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e411e0c5-cbd8-4a5a-95fc-c0739e2064b2 None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:52:27.520593 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-e411e0c5-cbd8-4a5a-95fc-c0739e2064b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:52:27.663168 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e411e0c5-cbd8-4a5a-95fc-c0739e2064b2 None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:52:27.702878 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:52:27.774903 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93124cb-8cc3-4b8b-9e8c-a16e5f5f31f2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4858bb8621854ec8aba836e7d9050498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:52:27.775438 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93124cb-8cc3-4b8b-9e8c-a16e5f5f31f2 service nova] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:52:27.775718 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93124cb-8cc3-4b8b-9e8c-a16e5f5f31f2 service nova] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:52:27.775990 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93124cb-8cc3-4b8b-9e8c-a16e5f5f31f2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:52:27.777648 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-c93124cb-8cc3-4b8b-9e8c-a16e5f5f31f2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:52:27.866010 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93124cb-8cc3-4b8b-9e8c-a16e5f5f31f2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:52:27.867658 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93124cb-8cc3-4b8b-9e8c-a16e5f5f31f2 service nova] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:52:27.885782 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 706/1419] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 14:52:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:52:28.758672 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07902354-e859-4983-b3de-cff39e5f7bfd None None] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:52:28.759229 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07902354-e859-4983-b3de-cff39e5f7bfd None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:52:28.759229 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07902354-e859-4983-b3de-cff39e5f7bfd None None] PATH_INFO: `/` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:52:28.760604 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 714/1420] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 1 14:52:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:52:28.765407 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204174c4-1a07-4a86-86ff-5af35d178ec0 None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:52:28.765820 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204174c4-1a07-4a86-86ff-5af35d178ec0 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:52:28.765947 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204174c4-1a07-4a86-86ff-5af35d178ec0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:52:28.819954 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:52:28.843151 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-204174c4-1a07-4a86-86ff-5af35d178ec0 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:52:28.844369 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-204174c4-1a07-4a86-86ff-5af35d178ec0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:52:28.937813 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 707/1421] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 1 14:52:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:52:29.089791 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07902354-e859-4983-b3de-cff39e5f7bfd None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:52:29.091886 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-07902354-e859-4983-b3de-cff39e5f7bfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:52:29.236018 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07902354-e859-4983-b3de-cff39e5f7bfd None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:52:29.248210 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:52:29.346594 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2860ca25-2fc8-4a1c-a155-b8f8120a57f0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4858bb8621854ec8aba836e7d9050498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:52:29.347196 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2860ca25-2fc8-4a1c-a155-b8f8120a57f0 service nova] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:52:29.347468 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2860ca25-2fc8-4a1c-a155-b8f8120a57f0 service nova] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:52:29.347730 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2860ca25-2fc8-4a1c-a155-b8f8120a57f0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:52:29.349318 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-2860ca25-2fc8-4a1c-a155-b8f8120a57f0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:52:29.437938 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2860ca25-2fc8-4a1c-a155-b8f8120a57f0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:52:29.439563 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2860ca25-2fc8-4a1c-a155-b8f8120a57f0 service nova] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:52:29.457182 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 715/1422] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 14:52:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:52:30.306858 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a325f4-522b-4e1f-ab9a-57d508cf3a21 None None] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:52:30.307402 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a325f4-522b-4e1f-ab9a-57d508cf3a21 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:52:30.307402 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a325f4-522b-4e1f-ab9a-57d508cf3a21 None None] PATH_INFO: `/` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:52:30.308747 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 708/1423] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 1 14:52:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:52:30.314103 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c26ca0-cc9f-452c-9095-6fa54558d2be None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:52:30.314525 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c26ca0-cc9f-452c-9095-6fa54558d2be None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:52:30.314684 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c26ca0-cc9f-452c-9095-6fa54558d2be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:52:30.335054 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:52:30.391311 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-24c26ca0-cc9f-452c-9095-6fa54558d2be None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:52:30.392525 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-24c26ca0-cc9f-452c-9095-6fa54558d2be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:52:30.486324 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 716/1424] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 1 14:52:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:52:30.636291 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a325f4-522b-4e1f-ab9a-57d508cf3a21 None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:52:30.637855 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-d2a325f4-522b-4e1f-ab9a-57d508cf3a21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:52:30.746064 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:52:30.781706 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a325f4-522b-4e1f-ab9a-57d508cf3a21 None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:52:30.892140 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccafa79f-b4b2-47f6-8f51-5922a11dbf91 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4858bb8621854ec8aba836e7d9050498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:52:30.892704 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccafa79f-b4b2-47f6-8f51-5922a11dbf91 service nova] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:52:30.892983 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccafa79f-b4b2-47f6-8f51-5922a11dbf91 service nova] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:52:30.893254 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccafa79f-b4b2-47f6-8f51-5922a11dbf91 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:52:30.894916 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-ccafa79f-b4b2-47f6-8f51-5922a11dbf91 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:52:30.969073 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:52:30.983808 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccafa79f-b4b2-47f6-8f51-5922a11dbf91 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:52:30.985403 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccafa79f-b4b2-47f6-8f51-5922a11dbf91 service nova] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:52:31.002946 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 709/1425] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 14:52:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:52:31.860120 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda546b2-6cf6-4917-a50d-7f513ec1614a None None] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:52:31.860628 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda546b2-6cf6-4917-a50d-7f513ec1614a None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:52:31.860628 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda546b2-6cf6-4917-a50d-7f513ec1614a None None] PATH_INFO: `/` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:52:31.861928 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 717/1426] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 1 14:52:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:52:31.866802 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ca49bf-132d-428a-bb8c-0cbe7ffb8cb0 None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:52:31.867222 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ca49bf-132d-428a-bb8c-0cbe7ffb8cb0 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:52:31.867353 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ca49bf-132d-428a-bb8c-0cbe7ffb8cb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:52:31.943435 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-f6ca49bf-132d-428a-bb8c-0cbe7ffb8cb0 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:52:31.944643 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-f6ca49bf-132d-428a-bb8c-0cbe7ffb8cb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:52:32.038404 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 710/1427] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 1 14:52:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:52:32.189387 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda546b2-6cf6-4917-a50d-7f513ec1614a None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:52:32.191145 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-dda546b2-6cf6-4917-a50d-7f513ec1614a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:52:32.272572 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:52:32.334875 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda546b2-6cf6-4917-a50d-7f513ec1614a None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:52:32.444218 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d86404-43ec-4f8d-b80e-1d2d66615630 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4858bb8621854ec8aba836e7d9050498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:52:32.444743 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d86404-43ec-4f8d-b80e-1d2d66615630 service nova] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:52:32.445042 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d86404-43ec-4f8d-b80e-1d2d66615630 service nova] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:52:32.445304 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d86404-43ec-4f8d-b80e-1d2d66615630 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:52:32.446906 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-26d86404-43ec-4f8d-b80e-1d2d66615630 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:52:32.494163 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:52:32.534835 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d86404-43ec-4f8d-b80e-1d2d66615630 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:52:32.536426 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d86404-43ec-4f8d-b80e-1d2d66615630 service nova] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:52:32.554479 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 718/1428] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 14:52:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:52:33.355363 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b208282e-2c8a-4a09-99d3-2ed163d23210 None None] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:52:33.355938 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b208282e-2c8a-4a09-99d3-2ed163d23210 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:52:33.355938 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b208282e-2c8a-4a09-99d3-2ed163d23210 None None] PATH_INFO: `/` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:52:33.357245 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 711/1429] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 1 14:52:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:52:33.362036 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb90d78-fbda-48b3-85b6-d4d255850e09 None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:52:33.362468 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb90d78-fbda-48b3-85b6-d4d255850e09 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:52:33.362608 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb90d78-fbda-48b3-85b6-d4d255850e09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:52:33.437384 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-7cb90d78-fbda-48b3-85b6-d4d255850e09 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:52:33.438633 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-7cb90d78-fbda-48b3-85b6-d4d255850e09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:52:33.532607 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 719/1430] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 1 14:52:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:52:33.683688 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b208282e-2c8a-4a09-99d3-2ed163d23210 None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:52:33.685833 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-b208282e-2c8a-4a09-99d3-2ed163d23210 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:52:33.733599 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:52:33.831749 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b208282e-2c8a-4a09-99d3-2ed163d23210 None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:52:33.940837 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265a7f44-7fd3-42cc-9493-1c44d506da1d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4858bb8621854ec8aba836e7d9050498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:52:33.941431 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265a7f44-7fd3-42cc-9493-1c44d506da1d service nova] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:52:33.941721 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265a7f44-7fd3-42cc-9493-1c44d506da1d service nova] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:52:33.941966 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265a7f44-7fd3-42cc-9493-1c44d506da1d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:52:33.943574 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-265a7f44-7fd3-42cc-9493-1c44d506da1d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:52:33.950298 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:52:34.030938 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265a7f44-7fd3-42cc-9493-1c44d506da1d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:52:34.032546 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265a7f44-7fd3-42cc-9493-1c44d506da1d service nova] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:52:34.050444 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 712/1431] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 14:52:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:52:34.842709 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd60118-6e70-4d4a-baf0-460fdb821b7b None None] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:52:34.843326 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd60118-6e70-4d4a-baf0-460fdb821b7b None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:52:34.843326 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd60118-6e70-4d4a-baf0-460fdb821b7b None None] PATH_INFO: `/` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:52:34.844607 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 720/1432] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 1 14:52:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:52:34.849373 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13983184-a5f1-41ed-b444-eacb5889b962 None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:52:34.849809 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13983184-a5f1-41ed-b444-eacb5889b962 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:52:34.849942 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13983184-a5f1-41ed-b444-eacb5889b962 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:52:34.926145 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-13983184-a5f1-41ed-b444-eacb5889b962 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:52:34.927323 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-13983184-a5f1-41ed-b444-eacb5889b962 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:52:34.993474 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:52:35.022352 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 713/1433] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 1 14:52:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:52:35.179473 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd60118-6e70-4d4a-baf0-460fdb821b7b None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:52:35.180903 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-5fd60118-6e70-4d4a-baf0-460fdb821b7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:52:35.198501 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:52:35.326748 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd60118-6e70-4d4a-baf0-460fdb821b7b None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:52:35.414149 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:52:35.439989 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b5a830-dc49-4d2d-8e33-016e552bbd97 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4858bb8621854ec8aba836e7d9050498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:52:35.440780 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b5a830-dc49-4d2d-8e33-016e552bbd97 service nova] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:52:35.441053 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b5a830-dc49-4d2d-8e33-016e552bbd97 service nova] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:52:35.441317 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b5a830-dc49-4d2d-8e33-016e552bbd97 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:52:35.442939 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-e1b5a830-dc49-4d2d-8e33-016e552bbd97 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:52:35.529835 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b5a830-dc49-4d2d-8e33-016e552bbd97 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:52:35.531432 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b5a830-dc49-4d2d-8e33-016e552bbd97 service nova] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:52:35.548574 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 721/1434] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 14:52:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:52:36.357276 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a94680-ae4b-4697-a610-8e98e0eb7d2b None None] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:52:36.357823 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a94680-ae4b-4697-a610-8e98e0eb7d2b None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:52:36.357823 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a94680-ae4b-4697-a610-8e98e0eb7d2b None None] PATH_INFO: `/` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:52:36.359178 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 714/1435] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 1 14:52:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:52:36.363938 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1847b33-9fce-4ddc-ba5d-a51e5f3696e4 None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:52:36.364334 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1847b33-9fce-4ddc-ba5d-a51e5f3696e4 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:52:36.364466 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1847b33-9fce-4ddc-ba5d-a51e5f3696e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:52:36.438955 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-a1847b33-9fce-4ddc-ba5d-a51e5f3696e4 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:52:36.440146 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-a1847b33-9fce-4ddc-ba5d-a51e5f3696e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:52:36.481308 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:52:36.535310 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 722/1436] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 1 14:52:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:52:36.692868 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a94680-ae4b-4697-a610-8e98e0eb7d2b None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:52:36.694416 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-22a94680-ae4b-4697-a610-8e98e0eb7d2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:52:36.837463 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a94680-ae4b-4697-a610-8e98e0eb7d2b None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:52:36.891810 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:52:36.949610 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6085bbd-e4ff-4036-b9a3-f883d1a72bbb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4858bb8621854ec8aba836e7d9050498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:52:36.950769 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6085bbd-e4ff-4036-b9a3-f883d1a72bbb service nova] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:52:36.951043 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6085bbd-e4ff-4036-b9a3-f883d1a72bbb service nova] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:52:36.951309 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6085bbd-e4ff-4036-b9a3-f883d1a72bbb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:52:36.952880 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-f6085bbd-e4ff-4036-b9a3-f883d1a72bbb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:52:37.041023 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6085bbd-e4ff-4036-b9a3-f883d1a72bbb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:52:37.042702 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6085bbd-e4ff-4036-b9a3-f883d1a72bbb service nova] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:52:37.061460 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 715/1437] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 14:52:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:52:37.855611 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b267f015-b63b-4d48-b559-5286bb7f9f4f None None] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:52:37.856140 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b267f015-b63b-4d48-b559-5286bb7f9f4f None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:52:37.856140 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b267f015-b63b-4d48-b559-5286bb7f9f4f None None] PATH_INFO: `/` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:52:37.858080 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 723/1438] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 1 14:52:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:52:37.863303 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ea520e-04f3-4208-a6af-ee86fe5e0a62 None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:52:37.863758 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ea520e-04f3-4208-a6af-ee86fe5e0a62 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:52:37.863864 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ea520e-04f3-4208-a6af-ee86fe5e0a62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:52:37.939603 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-b9ea520e-04f3-4208-a6af-ee86fe5e0a62 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:52:37.941074 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-b9ea520e-04f3-4208-a6af-ee86fe5e0a62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:52:37.943102 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:52:38.035540 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 716/1439] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 1 14:52:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:52:38.185548 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b267f015-b63b-4d48-b559-5286bb7f9f4f None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:52:38.187043 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-b267f015-b63b-4d48-b559-5286bb7f9f4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:52:38.334858 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b267f015-b63b-4d48-b559-5286bb7f9f4f None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:52:38.361393 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:52:38.446095 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a299f4ce-399c-40f6-b3cb-e56d7decf46a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4858bb8621854ec8aba836e7d9050498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:52:38.446655 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a299f4ce-399c-40f6-b3cb-e56d7decf46a service nova] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:52:38.446928 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a299f4ce-399c-40f6-b3cb-e56d7decf46a service nova] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:52:38.447193 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a299f4ce-399c-40f6-b3cb-e56d7decf46a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:52:38.448780 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-a299f4ce-399c-40f6-b3cb-e56d7decf46a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:52:38.535911 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a299f4ce-399c-40f6-b3cb-e56d7decf46a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:52:38.537483 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a299f4ce-399c-40f6-b3cb-e56d7decf46a service nova] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:52:38.555190 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 724/1440] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 14:52:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 14:52:40.379707 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cada0a13-083e-4599-b62b-b9b91eb454b6 None None] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:52:40.380276 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cada0a13-083e-4599-b62b-b9b91eb454b6 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:52:40.380276 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cada0a13-083e-4599-b62b-b9b91eb454b6 None None] PATH_INFO: `/` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:52:40.381587 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 717/1441] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 14:52:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:52:40.387061 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb332c6-39b7-49a0-9fd3-014de055dc2e None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:52:40.387467 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb332c6-39b7-49a0-9fd3-014de055dc2e None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:52:40.387606 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb332c6-39b7-49a0-9fd3-014de055dc2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:52:40.428362 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:52:40.453805 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-9bb332c6-39b7-49a0-9fd3-014de055dc2e None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:52:40.455067 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-9bb332c6-39b7-49a0-9fd3-014de055dc2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:52:40.549633 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 725/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 14:52:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:52:40.839701 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cada0a13-083e-4599-b62b-b9b91eb454b6 None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:52:40.841767 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-cada0a13-083e-4599-b62b-b9b91eb454b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:52:40.944546 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cada0a13-083e-4599-b62b-b9b91eb454b6 None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:52:40.976520 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:52:41.017285 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e4bd6c-b685-4c87-bc11-c9da2d135416 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2790149639aa418bbc4e4982e93eda44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:52:41.017886 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e4bd6c-b685-4c87-bc11-c9da2d135416 service neutron] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:52:41.018159 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e4bd6c-b685-4c87-bc11-c9da2d135416 service neutron] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:52:41.018466 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e4bd6c-b685-4c87-bc11-c9da2d135416 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:52:41.020080 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-67e4bd6c-b685-4c87-bc11-c9da2d135416 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:52:41.131090 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e4bd6c-b685-4c87-bc11-c9da2d135416 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:52:41.132533 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e4bd6c-b685-4c87-bc11-c9da2d135416 service neutron] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:52:41.150346 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 718/1443] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 14:52:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:52:43.174888 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd03b40c-fcc8-4cdc-8c20-b162b779b9c1 None None] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:52:43.175449 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd03b40c-fcc8-4cdc-8c20-b162b779b9c1 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:52:43.175449 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd03b40c-fcc8-4cdc-8c20-b162b779b9c1 None None] PATH_INFO: `/` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:52:43.176759 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 726/1444] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 14:52:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:52:43.181835 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545b580c-b9f4-4147-8644-b54caf8d76d3 None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:52:43.182266 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545b580c-b9f4-4147-8644-b54caf8d76d3 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:52:43.182420 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545b580c-b9f4-4147-8644-b54caf8d76d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:52:43.248237 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-545b580c-b9f4-4147-8644-b54caf8d76d3 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:52:43.249443 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-545b580c-b9f4-4147-8644-b54caf8d76d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:52:43.261846 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:52:43.345837 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 719/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 14:52:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:52:43.627912 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd03b40c-fcc8-4cdc-8c20-b162b779b9c1 None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:52:43.629511 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-cd03b40c-fcc8-4cdc-8c20-b162b779b9c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:52:43.733079 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd03b40c-fcc8-4cdc-8c20-b162b779b9c1 None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:52:43.744946 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:52:43.805250 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbf4a11-11ef-4da0-89bd-3b3975afd311 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2790149639aa418bbc4e4982e93eda44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:52:43.805845 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbf4a11-11ef-4da0-89bd-3b3975afd311 service neutron] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:52:43.806120 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbf4a11-11ef-4da0-89bd-3b3975afd311 service neutron] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:52:43.806417 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbf4a11-11ef-4da0-89bd-3b3975afd311 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:52:43.808010 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-5fbf4a11-11ef-4da0-89bd-3b3975afd311 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:52:43.914743 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbf4a11-11ef-4da0-89bd-3b3975afd311 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:52:43.916138 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbf4a11-11ef-4da0-89bd-3b3975afd311 service neutron] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:52:43.933996 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 727/1446] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 14:52:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:52:46.180646 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5c9e2d-a586-41e6-94be-8d45e4ac6c10 None None] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:52:46.181235 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5c9e2d-a586-41e6-94be-8d45e4ac6c10 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:52:46.181235 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5c9e2d-a586-41e6-94be-8d45e4ac6c10 None None] PATH_INFO: `/` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:52:46.182581 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 720/1447] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 14:52:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:52:46.187754 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d044be5a-d59a-4515-bedb-579faaa7a780 None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:52:46.188174 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d044be5a-d59a-4515-bedb-579faaa7a780 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:52:46.188290 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d044be5a-d59a-4515-bedb-579faaa7a780 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:52:46.235709 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:52:46.253348 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-d044be5a-d59a-4515-bedb-579faaa7a780 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:52:46.254571 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-d044be5a-d59a-4515-bedb-579faaa7a780 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:52:46.349364 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 728/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 14:52:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:52:46.630058 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5c9e2d-a586-41e6-94be-8d45e4ac6c10 None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:52:46.631635 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-4b5c9e2d-a586-41e6-94be-8d45e4ac6c10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:52:46.732373 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5c9e2d-a586-41e6-94be-8d45e4ac6c10 None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:52:46.771113 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:52:46.803266 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f319c602-c6f8-4d34-829b-bfe9f69aa913 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2790149639aa418bbc4e4982e93eda44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:52:46.803865 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f319c602-c6f8-4d34-829b-bfe9f69aa913 service neutron] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:52:46.804139 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f319c602-c6f8-4d34-829b-bfe9f69aa913 service neutron] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:52:46.804407 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f319c602-c6f8-4d34-829b-bfe9f69aa913 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:52:46.805995 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-f319c602-c6f8-4d34-829b-bfe9f69aa913 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:52:46.912231 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f319c602-c6f8-4d34-829b-bfe9f69aa913 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:52:46.913611 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f319c602-c6f8-4d34-829b-bfe9f69aa913 service neutron] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:52:46.931044 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 721/1449] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 14:52:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:52:48.928795 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8cd30c-d85c-4030-8d5d-885e4d62acc7 None None] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:52:48.929303 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8cd30c-d85c-4030-8d5d-885e4d62acc7 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:52:48.929303 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8cd30c-d85c-4030-8d5d-885e4d62acc7 None None] PATH_INFO: `/` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:52:48.930618 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 729/1450] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 14:52:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:52:48.936443 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ba176b-39ab-4f3e-8752-50dbd36899c8 None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:52:48.936892 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ba176b-39ab-4f3e-8752-50dbd36899c8 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:52:48.937003 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ba176b-39ab-4f3e-8752-50dbd36899c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:52:49.002446 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-a4ba176b-39ab-4f3e-8752-50dbd36899c8 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:52:49.003662 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-a4ba176b-39ab-4f3e-8752-50dbd36899c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:52:49.023691 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:52:49.100412 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 722/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 14:52:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:52:49.389518 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8cd30c-d85c-4030-8d5d-885e4d62acc7 None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:52:49.391101 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-ef8cd30c-d85c-4030-8d5d-885e4d62acc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:52:49.493413 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8cd30c-d85c-4030-8d5d-885e4d62acc7 None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:52:49.513225 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:52:49.566741 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec3c866-aa7b-4898-ad50-6adbb0ebf34f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2790149639aa418bbc4e4982e93eda44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:52:49.567342 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec3c866-aa7b-4898-ad50-6adbb0ebf34f service neutron] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:52:49.567623 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec3c866-aa7b-4898-ad50-6adbb0ebf34f service neutron] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:52:49.567893 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec3c866-aa7b-4898-ad50-6adbb0ebf34f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:52:49.569507 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-3ec3c866-aa7b-4898-ad50-6adbb0ebf34f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:52:49.680497 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec3c866-aa7b-4898-ad50-6adbb0ebf34f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:52:49.681947 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec3c866-aa7b-4898-ad50-6adbb0ebf34f service neutron] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:52:49.700907 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 730/1452] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 14:52:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:52:51.632256 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa78526-4ed8-47d0-8943-03909db1df5b None None] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:52:51.632745 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa78526-4ed8-47d0-8943-03909db1df5b None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:52:51.632842 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa78526-4ed8-47d0-8943-03909db1df5b None None] PATH_INFO: `/` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:52:51.634124 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 723/1453] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 14:52:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:52:51.639287 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4633c2d5-3905-4651-a159-93191f86a0ef None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:52:51.639731 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4633c2d5-3905-4651-a159-93191f86a0ef None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:52:51.639851 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4633c2d5-3905-4651-a159-93191f86a0ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:52:51.702722 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:52:51.705527 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-4633c2d5-3905-4651-a159-93191f86a0ef None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:52:51.707368 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-4633c2d5-3905-4651-a159-93191f86a0ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:52:51.801366 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 731/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 14:52:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:52:52.085917 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa78526-4ed8-47d0-8943-03909db1df5b None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:52:52.087515 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-daa78526-4ed8-47d0-8943-03909db1df5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:52:52.191260 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa78526-4ed8-47d0-8943-03909db1df5b None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:52:52.239682 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:52:52.264187 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4c52e4-5415-4ef1-8211-be41598e688b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2790149639aa418bbc4e4982e93eda44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:52:52.264795 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4c52e4-5415-4ef1-8211-be41598e688b service neutron] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:52:52.265076 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4c52e4-5415-4ef1-8211-be41598e688b service neutron] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:52:52.265351 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4c52e4-5415-4ef1-8211-be41598e688b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:52:52.266973 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-4c4c52e4-5415-4ef1-8211-be41598e688b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:52:52.374354 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4c52e4-5415-4ef1-8211-be41598e688b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:52:52.375742 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4c52e4-5415-4ef1-8211-be41598e688b service neutron] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:52:52.393881 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 724/1455] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 14:52:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:52:54.315820 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd6c282-02a2-421d-9688-2cec844ef047 None None] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:52:54.316422 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd6c282-02a2-421d-9688-2cec844ef047 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:52:54.316422 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd6c282-02a2-421d-9688-2cec844ef047 None None] PATH_INFO: `/` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:52:54.317752 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 732/1456] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 14:52:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:52:54.322828 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de60d06f-44b2-4299-9516-bd22348ed08f None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:52:54.323277 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de60d06f-44b2-4299-9516-bd22348ed08f None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:52:54.323381 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de60d06f-44b2-4299-9516-bd22348ed08f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:52:54.388293 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-de60d06f-44b2-4299-9516-bd22348ed08f None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:52:54.389491 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-de60d06f-44b2-4299-9516-bd22348ed08f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:52:54.417347 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:52:54.483750 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 725/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 14:52:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:52:54.774635 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd6c282-02a2-421d-9688-2cec844ef047 None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:52:54.776174 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-9bd6c282-02a2-421d-9688-2cec844ef047 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:52:54.879919 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd6c282-02a2-421d-9688-2cec844ef047 None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:52:54.907464 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:52:54.952835 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98963745-c7e5-408c-a0c4-331d13b9d303 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2790149639aa418bbc4e4982e93eda44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:52:54.953367 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98963745-c7e5-408c-a0c4-331d13b9d303 service neutron] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:52:54.953646 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98963745-c7e5-408c-a0c4-331d13b9d303 service neutron] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:52:54.953913 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98963745-c7e5-408c-a0c4-331d13b9d303 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:52:54.955542 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-98963745-c7e5-408c-a0c4-331d13b9d303 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:52:55.063392 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98963745-c7e5-408c-a0c4-331d13b9d303 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:52:55.065105 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98963745-c7e5-408c-a0c4-331d13b9d303 service neutron] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:52:55.082958 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 733/1458] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 14:52:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:52:57.002827 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e039d2-18e3-481c-992d-e8f0631dc2e7 None None] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:52:57.003346 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e039d2-18e3-481c-992d-e8f0631dc2e7 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:52:57.003346 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e039d2-18e3-481c-992d-e8f0631dc2e7 None None] PATH_INFO: `/` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:52:57.004633 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 726/1459] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 14:52:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:52:57.009940 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933e3056-b625-48d3-a7d8-4c4fa3bc514c None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:52:57.010390 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933e3056-b625-48d3-a7d8-4c4fa3bc514c None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:52:57.010488 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933e3056-b625-48d3-a7d8-4c4fa3bc514c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:52:57.076096 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-933e3056-b625-48d3-a7d8-4c4fa3bc514c None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:52:57.077305 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-933e3056-b625-48d3-a7d8-4c4fa3bc514c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:52:57.085407 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:52:57.171977 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 734/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 14:52:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:52:57.454959 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e039d2-18e3-481c-992d-e8f0631dc2e7 None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:52:57.456566 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-f7e039d2-18e3-481c-992d-e8f0631dc2e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:52:57.557644 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e039d2-18e3-481c-992d-e8f0631dc2e7 None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:52:57.614408 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:52:57.633028 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007caf13-34ab-4570-85d5-b4f45d04799a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2790149639aa418bbc4e4982e93eda44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:52:57.633634 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007caf13-34ab-4570-85d5-b4f45d04799a service neutron] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:52:57.633914 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007caf13-34ab-4570-85d5-b4f45d04799a service neutron] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:52:57.634185 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007caf13-34ab-4570-85d5-b4f45d04799a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:52:57.635810 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-007caf13-34ab-4570-85d5-b4f45d04799a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:52:57.743449 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007caf13-34ab-4570-85d5-b4f45d04799a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:52:57.744832 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007caf13-34ab-4570-85d5-b4f45d04799a service neutron] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:52:57.762846 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 727/1461] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 14:52:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:52:59.684986 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ee1d74-f69a-4b97-8ef4-190c8a766144 None None] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:52:59.685457 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ee1d74-f69a-4b97-8ef4-190c8a766144 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:52:59.685568 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ee1d74-f69a-4b97-8ef4-190c8a766144 None None] PATH_INFO: `/` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:52:59.687016 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 735/1462] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 14:52:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:52:59.692581 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e303407-1336-4a0c-b593-034603060333 None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:52:59.693018 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e303407-1336-4a0c-b593-034603060333 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:52:59.693128 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e303407-1336-4a0c-b593-034603060333 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:52:59.758736 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-4e303407-1336-4a0c-b593-034603060333 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:52:59.759973 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-4e303407-1336-4a0c-b593-034603060333 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:52:59.797672 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:52:59.855115 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 728/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 14:52:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:53:00.137938 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ee1d74-f69a-4b97-8ef4-190c8a766144 None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:53:00.139547 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-22ee1d74-f69a-4b97-8ef4-190c8a766144 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:53:00.240642 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ee1d74-f69a-4b97-8ef4-190c8a766144 None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:53:00.276600 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:53:00.312311 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdda4ba-26ec-406f-b895-ec8aa7ccbf3f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2790149639aa418bbc4e4982e93eda44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:53:00.312909 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdda4ba-26ec-406f-b895-ec8aa7ccbf3f service neutron] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:53:00.313186 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdda4ba-26ec-406f-b895-ec8aa7ccbf3f service neutron] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:53:00.313455 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdda4ba-26ec-406f-b895-ec8aa7ccbf3f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:53:00.315334 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-9cdda4ba-26ec-406f-b895-ec8aa7ccbf3f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:53:00.421124 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdda4ba-26ec-406f-b895-ec8aa7ccbf3f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:53:00.422509 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdda4ba-26ec-406f-b895-ec8aa7ccbf3f service neutron] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:53:00.441633 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 736/1464] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 14:53:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:53:02.327646 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996a0ea9-d03a-4bbe-9546-fb86430abadc None None] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:53:02.328258 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996a0ea9-d03a-4bbe-9546-fb86430abadc None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:53:02.328258 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996a0ea9-d03a-4bbe-9546-fb86430abadc None None] PATH_INFO: `/` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:53:02.329517 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 729/1465] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 14:53:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:53:02.334697 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581dd133-611a-425e-823a-096805e9409b None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:53:02.335126 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581dd133-611a-425e-823a-096805e9409b None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:53:02.335279 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581dd133-611a-425e-823a-096805e9409b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:53:02.400406 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-581dd133-611a-425e-823a-096805e9409b None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:53:02.401612 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-581dd133-611a-425e-823a-096805e9409b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:53:02.419741 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:53:02.496886 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 737/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 14:53:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 14:53:02.776111 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996a0ea9-d03a-4bbe-9546-fb86430abadc None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:53:02.777680 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-996a0ea9-d03a-4bbe-9546-fb86430abadc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:53:02.880731 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996a0ea9-d03a-4bbe-9546-fb86430abadc None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:53:02.945122 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:53:02.952098 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d653da0-d5e4-4472-9765-017e114cf48b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2790149639aa418bbc4e4982e93eda44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:53:02.952699 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d653da0-d5e4-4472-9765-017e114cf48b service neutron] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:53:02.953004 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d653da0-d5e4-4472-9765-017e114cf48b service neutron] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:53:02.953275 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d653da0-d5e4-4472-9765-017e114cf48b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:53:02.954895 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-9d653da0-d5e4-4472-9765-017e114cf48b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:53:03.061825 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d653da0-d5e4-4472-9765-017e114cf48b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:53:03.063313 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d653da0-d5e4-4472-9765-017e114cf48b service neutron] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:53:03.080929 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 730/1467] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 14:53:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:53:04.970725 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feafa21c-4690-4c6c-89dc-c0c5a3f64f12 None None] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:53:04.971283 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feafa21c-4690-4c6c-89dc-c0c5a3f64f12 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:53:04.971283 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feafa21c-4690-4c6c-89dc-c0c5a3f64f12 None None] PATH_INFO: `/` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:53:04.972587 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 738/1468] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 14:53:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:53:04.977639 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddc95e9-1e84-4cef-be2b-27372e5378fb None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:53:04.978059 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddc95e9-1e84-4cef-be2b-27372e5378fb None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:53:04.978187 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddc95e9-1e84-4cef-be2b-27372e5378fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:53:05.043130 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-4ddc95e9-1e84-4cef-be2b-27372e5378fb None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:53:05.044330 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-4ddc95e9-1e84-4cef-be2b-27372e5378fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:53:05.091214 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:53:05.138803 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 731/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 14:53:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:53:05.420081 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feafa21c-4690-4c6c-89dc-c0c5a3f64f12 None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:53:05.421827 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-feafa21c-4690-4c6c-89dc-c0c5a3f64f12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:53:05.524434 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feafa21c-4690-4c6c-89dc-c0c5a3f64f12 None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:53:05.570016 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:53:05.596379 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71faa104-9313-4e6c-93d7-12688a59d759 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2790149639aa418bbc4e4982e93eda44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:53:05.596970 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71faa104-9313-4e6c-93d7-12688a59d759 service neutron] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:53:05.597243 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71faa104-9313-4e6c-93d7-12688a59d759 service neutron] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:53:05.597507 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71faa104-9313-4e6c-93d7-12688a59d759 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:53:05.599129 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-71faa104-9313-4e6c-93d7-12688a59d759 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:53:05.706979 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71faa104-9313-4e6c-93d7-12688a59d759 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:53:05.708418 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71faa104-9313-4e6c-93d7-12688a59d759 service neutron] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:53:05.726427 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 739/1470] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 14:53:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:53:15.136528 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=22 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:53:15.724155 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=66 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:53:24.943134 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcbf524-65fe-4209-832d-328d265feeee None None] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:53:24.943671 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcbf524-65fe-4209-832d-328d265feeee None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:53:24.943671 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcbf524-65fe-4209-832d-328d265feeee None None] PATH_INFO: `/` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:53:24.944983 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 732/1471] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 1 14:53:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:53:24.950891 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4df5823-5741-436b-b1fa-8fb0ef863296 None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:53:24.951310 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4df5823-5741-436b-b1fa-8fb0ef863296 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:53:24.951420 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4df5823-5741-436b-b1fa-8fb0ef863296 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:53:25.018411 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-e4df5823-5741-436b-b1fa-8fb0ef863296 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:53:25.019617 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-e4df5823-5741-436b-b1fa-8fb0ef863296 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:53:25.113438 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 740/1472] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 1 14:53:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:53:25.406705 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31693ffb-b01b-4712-8f4a-c676e9f86606 None None] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:53:25.407000 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31693ffb-b01b-4712-8f4a-c676e9f86606 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:53:25.407266 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31693ffb-b01b-4712-8f4a-c676e9f86606 None None] PATH_INFO: `/` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:53:25.408605 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 733/1473] 10.222.0.33 () {54 vars in 967 bytes} [Thu Sep 1 14:53:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:53:25.413383 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39967f50-d759-4ea6-ae36-1e5eca79223a None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:53:25.413664 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39967f50-d759-4ea6-ae36-1e5eca79223a None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:53:25.413933 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39967f50-d759-4ea6-ae36-1e5eca79223a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:53:25.489050 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-39967f50-d759-4ea6-ae36-1e5eca79223a None None] MFA Rules not processed for user `3069f72ce98049b99c9d98229a62bb51`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:53:25.490285 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-39967f50-d759-4ea6-ae36-1e5eca79223a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:53:25.511107 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:53:25.566205 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 741/1474] 10.222.0.33 () {58 vars in 1064 bytes} [Thu Sep 1 14:53:25 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:53:25.573721 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31693ffb-b01b-4712-8f4a-c676e9f86606 None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:53:25.575273 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-31693ffb-b01b-4712-8f4a-c676e9f86606 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:53:25.679016 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31693ffb-b01b-4712-8f4a-c676e9f86606 None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:53:25.750533 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f274fb2-140a-4cb6-be12-bb639bfc37e7 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3069f72ce98049b99c9d98229a62bb51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:53:25.751133 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f274fb2-140a-4cb6-be12-bb639bfc37e7 service glance] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:53:25.751406 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f274fb2-140a-4cb6-be12-bb639bfc37e7 service glance] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:53:25.751671 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f274fb2-140a-4cb6-be12-bb639bfc37e7 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:53:25.753252 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-7f274fb2-140a-4cb6-be12-bb639bfc37e7 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:53:25.805401 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:53:25.859259 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f274fb2-140a-4cb6-be12-bb639bfc37e7 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:53:25.860707 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f274fb2-140a-4cb6-be12-bb639bfc37e7 service glance] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:53:25.879884 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 734/1475] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 14:53:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:53:27.844891 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7c78a7-3ed8-4a34-bbed-eb1b31bfb16b None None] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:53:27.845450 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7c78a7-3ed8-4a34-bbed-eb1b31bfb16b None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:53:27.845450 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7c78a7-3ed8-4a34-bbed-eb1b31bfb16b None None] PATH_INFO: `/` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:53:27.846816 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 742/1476] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 1 14:53:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:53:27.851826 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddae488-744b-4698-8fd2-861cca61bde6 None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:53:27.852276 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddae488-744b-4698-8fd2-861cca61bde6 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:53:27.852381 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddae488-744b-4698-8fd2-861cca61bde6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:53:27.916786 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-5ddae488-744b-4698-8fd2-861cca61bde6 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:53:27.917907 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-5ddae488-744b-4698-8fd2-861cca61bde6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:53:28.013650 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 735/1477] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 1 14:53:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:53:28.281414 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7c78a7-3ed8-4a34-bbed-eb1b31bfb16b None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:53:28.282951 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-5d7c78a7-3ed8-4a34-bbed-eb1b31bfb16b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:53:28.406207 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7c78a7-3ed8-4a34-bbed-eb1b31bfb16b None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:53:28.454216 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:53:28.498697 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9daf8d-28aa-495f-953d-78b1177bd867 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:53:28.499291 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9daf8d-28aa-495f-953d-78b1177bd867 admin admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:53:28.499564 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9daf8d-28aa-495f-953d-78b1177bd867 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:53:28.499824 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9daf8d-28aa-495f-953d-78b1177bd867 admin admin] PATH_INFO: `/v3/projects` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:53:28.500449 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9daf8d-28aa-495f-953d-78b1177bd867 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:53:28.502140 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: WARNING py.warnings [None req-fb9daf8d-28aa-495f-953d-78b1177bd867 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 01 14:53:28.502140 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: warnings.warn(msg) Sep 01 14:53:28.502140 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: Sep 01 14:53:28.503391 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9daf8d-28aa-495f-953d-78b1177bd867 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:53:28.521839 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 743/1478] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 1 14:53:28 2022] GET /identity/v3/projects => generated 2553 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:53:30.428097 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784b752f-004f-4387-b972-0ecefba4a481 None None] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:53:30.428663 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784b752f-004f-4387-b972-0ecefba4a481 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:53:30.428663 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784b752f-004f-4387-b972-0ecefba4a481 None None] PATH_INFO: `/` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:53:30.429976 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 736/1479] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 1 14:53:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:53:30.435912 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15354d0a-46f3-4ea2-90c5-a7b8cf5f19a6 None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:53:30.436336 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15354d0a-46f3-4ea2-90c5-a7b8cf5f19a6 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:53:30.436513 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15354d0a-46f3-4ea2-90c5-a7b8cf5f19a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:53:30.501022 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-15354d0a-46f3-4ea2-90c5-a7b8cf5f19a6 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:53:30.502212 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-15354d0a-46f3-4ea2-90c5-a7b8cf5f19a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:53:30.595935 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 744/1480] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 1 14:53:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:53:30.889231 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784b752f-004f-4387-b972-0ecefba4a481 None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:53:30.890965 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-784b752f-004f-4387-b972-0ecefba4a481 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:53:30.900846 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:53:31.033408 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784b752f-004f-4387-b972-0ecefba4a481 None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:53:31.111894 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:53:31.144789 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22dc3093-3a39-4952-a45e-03768306afdc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4858bb8621854ec8aba836e7d9050498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:53:31.145326 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dc3093-3a39-4952-a45e-03768306afdc service nova] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:53:31.145607 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dc3093-3a39-4952-a45e-03768306afdc service nova] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:53:31.145880 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dc3093-3a39-4952-a45e-03768306afdc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:53:31.147559 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-22dc3093-3a39-4952-a45e-03768306afdc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:53:31.235709 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22dc3093-3a39-4952-a45e-03768306afdc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:53:31.237309 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22dc3093-3a39-4952-a45e-03768306afdc service nova] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:53:31.254785 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 737/1481] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 14:53:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:53:33.322050 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2699a27-37fc-401d-84b6-d09352ab5379 None None] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:53:33.322658 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2699a27-37fc-401d-84b6-d09352ab5379 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:53:33.322658 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2699a27-37fc-401d-84b6-d09352ab5379 None None] PATH_INFO: `/` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:53:33.323972 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 745/1482] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 1 14:53:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:53:33.329085 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9d2aaf-3249-4bb9-a448-0809b9c580e3 None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:53:33.329510 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9d2aaf-3249-4bb9-a448-0809b9c580e3 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:53:33.329641 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9d2aaf-3249-4bb9-a448-0809b9c580e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:53:33.395947 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-1f9d2aaf-3249-4bb9-a448-0809b9c580e3 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:53:33.397175 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-1f9d2aaf-3249-4bb9-a448-0809b9c580e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:53:33.444665 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:53:33.497430 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 738/1483] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 1 14:53:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:53:33.780772 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2699a27-37fc-401d-84b6-d09352ab5379 None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:53:33.782347 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-b2699a27-37fc-401d-84b6-d09352ab5379 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:53:33.805217 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:53:33.883432 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2699a27-37fc-401d-84b6-d09352ab5379 None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:53:33.955602 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14161686-93ef-48a3-8f5f-7b34bca36701 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2790149639aa418bbc4e4982e93eda44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:53:33.956203 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14161686-93ef-48a3-8f5f-7b34bca36701 service neutron] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:53:33.956479 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14161686-93ef-48a3-8f5f-7b34bca36701 service neutron] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:53:33.956747 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14161686-93ef-48a3-8f5f-7b34bca36701 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:53:33.958359 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-14161686-93ef-48a3-8f5f-7b34bca36701 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:53:34.030603 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:53:34.067935 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14161686-93ef-48a3-8f5f-7b34bca36701 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:53:34.069844 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14161686-93ef-48a3-8f5f-7b34bca36701 service neutron] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:53:34.093117 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 746/1484] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 14:53:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:53:36.034330 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a7a2b7-a373-4f23-a5a8-06a709037003 None None] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:53:36.034933 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a7a2b7-a373-4f23-a5a8-06a709037003 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:53:36.034933 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a7a2b7-a373-4f23-a5a8-06a709037003 None None] PATH_INFO: `/` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:53:36.036241 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 739/1485] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 1 14:53:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:53:36.041897 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ca6f0e-ca1f-4577-94c6-f4823e2dc317 None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:53:36.042358 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ca6f0e-ca1f-4577-94c6-f4823e2dc317 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:53:36.042468 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ca6f0e-ca1f-4577-94c6-f4823e2dc317 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:53:36.107562 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-d9ca6f0e-ca1f-4577-94c6-f4823e2dc317 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:53:36.108781 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-d9ca6f0e-ca1f-4577-94c6-f4823e2dc317 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:53:36.203353 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 747/1486] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 1 14:53:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:53:36.497959 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a7a2b7-a373-4f23-a5a8-06a709037003 None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:53:36.499508 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-56a7a2b7-a373-4f23-a5a8-06a709037003 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:53:36.599155 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a7a2b7-a373-4f23-a5a8-06a709037003 None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:53:36.669015 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:53:36.671003 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f10dec-034b-4313-a6a7-92ccd2af6bf8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2790149639aa418bbc4e4982e93eda44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:53:36.671792 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f10dec-034b-4313-a6a7-92ccd2af6bf8 service neutron] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:53:36.672431 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f10dec-034b-4313-a6a7-92ccd2af6bf8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:53:36.672893 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f10dec-034b-4313-a6a7-92ccd2af6bf8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:53:36.674720 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-e8f10dec-034b-4313-a6a7-92ccd2af6bf8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:53:36.782431 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f10dec-034b-4313-a6a7-92ccd2af6bf8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:53:36.783820 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f10dec-034b-4313-a6a7-92ccd2af6bf8 service neutron] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:53:36.801258 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 740/1487] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 14:53:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:53:38.784194 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73c7cb9-ba7d-446c-8972-23a7ed3f5c43 None None] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:53:38.784769 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73c7cb9-ba7d-446c-8972-23a7ed3f5c43 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:53:38.784769 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73c7cb9-ba7d-446c-8972-23a7ed3f5c43 None None] PATH_INFO: `/` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:53:38.786594 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 748/1488] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 1 14:53:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:53:38.792488 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c37d8b4-be1a-4a7f-89aa-29e8be549a3f None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:53:38.792909 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c37d8b4-be1a-4a7f-89aa-29e8be549a3f None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:53:38.793041 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c37d8b4-be1a-4a7f-89aa-29e8be549a3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:53:38.858909 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-8c37d8b4-be1a-4a7f-89aa-29e8be549a3f None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:53:38.860098 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-8c37d8b4-be1a-4a7f-89aa-29e8be549a3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:53:38.912679 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:53:38.956488 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 741/1489] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 1 14:53:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:53:39.263375 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73c7cb9-ba7d-446c-8972-23a7ed3f5c43 None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:53:39.264904 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-e73c7cb9-ba7d-446c-8972-23a7ed3f5c43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:53:39.300533 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:53:39.367793 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73c7cb9-ba7d-446c-8972-23a7ed3f5c43 None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:53:39.439250 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747a7aa1-90ea-4728-bfae-29ac1f8500f4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2790149639aa418bbc4e4982e93eda44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:53:39.439846 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747a7aa1-90ea-4728-bfae-29ac1f8500f4 service neutron] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:53:39.440120 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747a7aa1-90ea-4728-bfae-29ac1f8500f4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:53:39.440383 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747a7aa1-90ea-4728-bfae-29ac1f8500f4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:53:39.441958 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-747a7aa1-90ea-4728-bfae-29ac1f8500f4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:53:39.520477 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:53:39.549861 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747a7aa1-90ea-4728-bfae-29ac1f8500f4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:53:39.551327 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747a7aa1-90ea-4728-bfae-29ac1f8500f4 service neutron] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:53:39.569952 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 749/1490] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 14:53:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:53:41.671678 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f3c309-9193-443f-8214-8a1f196039bf None None] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:53:41.672126 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f3c309-9193-443f-8214-8a1f196039bf None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:53:41.672224 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f3c309-9193-443f-8214-8a1f196039bf None None] PATH_INFO: `/` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:53:41.673536 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 742/1491] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 1 14:53:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:53:41.679486 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258aece5-7fc5-4a65-b2ea-6bd8bfb79149 None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:53:41.679898 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258aece5-7fc5-4a65-b2ea-6bd8bfb79149 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:53:41.680031 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258aece5-7fc5-4a65-b2ea-6bd8bfb79149 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:53:41.746217 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-258aece5-7fc5-4a65-b2ea-6bd8bfb79149 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:53:41.747469 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-258aece5-7fc5-4a65-b2ea-6bd8bfb79149 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:53:41.842405 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 750/1492] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 1 14:53:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:53:42.127466 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f3c309-9193-443f-8214-8a1f196039bf None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:53:42.129013 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-e3f3c309-9193-443f-8214-8a1f196039bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:53:42.230649 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f3c309-9193-443f-8214-8a1f196039bf None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:53:42.301531 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c08414-f6c8-4c13-b6a3-b77ba55799d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2790149639aa418bbc4e4982e93eda44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:53:42.302134 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c08414-f6c8-4c13-b6a3-b77ba55799d7 service neutron] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:53:42.302441 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c08414-f6c8-4c13-b6a3-b77ba55799d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:53:42.302718 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c08414-f6c8-4c13-b6a3-b77ba55799d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:53:42.304310 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-47c08414-f6c8-4c13-b6a3-b77ba55799d7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:53:42.314076 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:53:42.412239 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c08414-f6c8-4c13-b6a3-b77ba55799d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:53:42.413624 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c08414-f6c8-4c13-b6a3-b77ba55799d7 service neutron] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:53:42.431747 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 743/1493] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 14:53:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:53:51.840597 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=82 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:53:52.428986 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=50 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:54:54.221709 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea3ee6c-3a0b-42bd-a5a0-db6aed6de842 None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:54:54.222343 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea3ee6c-3a0b-42bd-a5a0-db6aed6de842 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:54:54.222343 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea3ee6c-3a0b-42bd-a5a0-db6aed6de842 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:54:54.295920 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-9ea3ee6c-3a0b-42bd-a5a0-db6aed6de842 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:54:54.297123 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-9ea3ee6c-3a0b-42bd-a5a0-db6aed6de842 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:54:54.392351 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 751/1494] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 14:54:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:54:54.400291 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f449c26a-a806-4a70-a25e-722cec066de1 None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:54:54.400806 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f449c26a-a806-4a70-a25e-722cec066de1 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:54:54.400928 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f449c26a-a806-4a70-a25e-722cec066de1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:54:54.474567 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-f449c26a-a806-4a70-a25e-722cec066de1 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:54:54.475695 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-f449c26a-a806-4a70-a25e-722cec066de1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:54:54.570060 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 744/1495] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 14:54:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:54:54.574631 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea3ee6c-3a0b-42bd-a5a0-db6aed6de842 None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:54:54.575832 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-9ea3ee6c-3a0b-42bd-a5a0-db6aed6de842 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:54:54.587003 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea3ee6c-3a0b-42bd-a5a0-db6aed6de842 None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:54:54.646581 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:54:54.681212 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e492e7-9523-43f8-87f7-b0b33271a11e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:54:54.681811 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e492e7-9523-43f8-87f7-b0b33271a11e admin admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:54:54.682082 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e492e7-9523-43f8-87f7-b0b33271a11e admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:54:54.682376 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e492e7-9523-43f8-87f7-b0b33271a11e admin admin] PATH_INFO: `/v3/domains` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:54:54.683012 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e492e7-9523-43f8-87f7-b0b33271a11e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:54:54.683304 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e492e7-9523-43f8-87f7-b0b33271a11e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:54:54.684305 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: WARNING py.warnings [None req-24e492e7-9523-43f8-87f7-b0b33271a11e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 01 14:54:54.684305 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: warnings.warn(msg) Sep 01 14:54:54.684305 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: Sep 01 14:54:54.685182 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e492e7-9523-43f8-87f7-b0b33271a11e admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:54:54.696333 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 752/1496] 10.222.0.33 () {58 vars in 1153 bytes} [Thu Sep 1 14:54:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:54:54.704606 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f449c26a-a806-4a70-a25e-722cec066de1 None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:54:54.713579 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f449c26a-a806-4a70-a25e-722cec066de1 None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:54:54.890372 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:54:54.923703 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26efa45e-4d88-461e-b90d-1a149ba5bb27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:54:54.924301 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26efa45e-4d88-461e-b90d-1a149ba5bb27 admin admin] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:54:54.924576 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26efa45e-4d88-461e-b90d-1a149ba5bb27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:54:54.924843 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26efa45e-4d88-461e-b90d-1a149ba5bb27 admin admin] PATH_INFO: `/v3/projects` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:54:54.925474 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26efa45e-4d88-461e-b90d-1a149ba5bb27 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:54:54.927245 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: WARNING py.warnings [None req-26efa45e-4d88-461e-b90d-1a149ba5bb27 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 01 14:54:54.927245 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: warnings.warn(msg) Sep 01 14:54:54.927245 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: Sep 01 14:54:54.928442 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26efa45e-4d88-461e-b90d-1a149ba5bb27 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:54:54.983396 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 745/1497] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Sep 1 14:54:54 2022] POST /identity/v3/projects => generated 379 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:54:54.987659 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e492e7-9523-43f8-87f7-b0b33271a11e admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:54:54.989183 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-24e492e7-9523-43f8-87f7-b0b33271a11e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:54:55.021912 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e492e7-9523-43f8-87f7-b0b33271a11e admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:54:55.116438 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828b2867-7183-4459-a9c7-e738d5117477 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:54:55.117035 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828b2867-7183-4459-a9c7-e738d5117477 admin admin] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:54:55.117310 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828b2867-7183-4459-a9c7-e738d5117477 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:54:55.117999 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828b2867-7183-4459-a9c7-e738d5117477 admin admin] PATH_INFO: `/v3/users` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:54:55.118668 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828b2867-7183-4459-a9c7-e738d5117477 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:54:55.119679 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: WARNING py.warnings [None req-828b2867-7183-4459-a9c7-e738d5117477 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 01 14:54:55.119679 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: warnings.warn(msg) Sep 01 14:54:55.119679 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: Sep 01 14:54:55.120890 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828b2867-7183-4459-a9c7-e738d5117477 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:54:55.161557 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 753/1498] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Sep 1 14:54:54 2022] POST /identity/v3/users => generated 290 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 14:54:55.166174 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26efa45e-4d88-461e-b90d-1a149ba5bb27 admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:54:55.167724 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-26efa45e-4d88-461e-b90d-1a149ba5bb27 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:54:55.228455 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26efa45e-4d88-461e-b90d-1a149ba5bb27 admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:54:55.308786 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:54:55.319980 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc472d7a-8d4b-435e-a9ba-bf9650c21153 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:54:55.321099 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc472d7a-8d4b-435e-a9ba-bf9650c21153 admin admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:54:55.321377 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc472d7a-8d4b-435e-a9ba-bf9650c21153 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:54:55.321648 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc472d7a-8d4b-435e-a9ba-bf9650c21153 admin admin] PATH_INFO: `/v3/roles` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:54:55.322325 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc472d7a-8d4b-435e-a9ba-bf9650c21153 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:54:55.323356 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: WARNING py.warnings [None req-cc472d7a-8d4b-435e-a9ba-bf9650c21153 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 01 14:54:55.323356 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: warnings.warn(msg) Sep 01 14:54:55.323356 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: Sep 01 14:54:55.324237 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc472d7a-8d4b-435e-a9ba-bf9650c21153 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:54:55.337447 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 746/1499] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 1 14:54:55 2022] GET /identity/v3/roles => generated 1420 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:54:55.341599 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828b2867-7183-4459-a9c7-e738d5117477 admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:54:55.343178 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-828b2867-7183-4459-a9c7-e738d5117477 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:54:55.354504 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828b2867-7183-4459-a9c7-e738d5117477 admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:54:55.379094 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:54:55.451201 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb14fae-2c81-4a0a-b31b-5ffcabb35fc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:54:55.451731 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb14fae-2c81-4a0a-b31b-5ffcabb35fc1 admin admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:54:55.452004 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb14fae-2c81-4a0a-b31b-5ffcabb35fc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:54:55.452268 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb14fae-2c81-4a0a-b31b-5ffcabb35fc1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:54:55.452906 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb14fae-2c81-4a0a-b31b-5ffcabb35fc1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:54:55.453907 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: WARNING py.warnings [None req-deb14fae-2c81-4a0a-b31b-5ffcabb35fc1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 01 14:54:55.453907 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: warnings.warn(msg) Sep 01 14:54:55.453907 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: Sep 01 14:54:55.454819 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb14fae-2c81-4a0a-b31b-5ffcabb35fc1 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:54:55.467290 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 754/1500] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 1 14:54:55 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:54:55.471429 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc472d7a-8d4b-435e-a9ba-bf9650c21153 admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:54:55.472934 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-cc472d7a-8d4b-435e-a9ba-bf9650c21153 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:54:55.504583 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc472d7a-8d4b-435e-a9ba-bf9650c21153 admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:54:55.595706 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be6b121-ab03-427a-bf06-acfcd96f3aec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:54:55.596360 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be6b121-ab03-427a-bf06-acfcd96f3aec admin admin] REQUEST_METHOD: `PUT` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:54:55.596640 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be6b121-ab03-427a-bf06-acfcd96f3aec admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:54:55.596920 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be6b121-ab03-427a-bf06-acfcd96f3aec admin admin] PATH_INFO: `/v3/projects/d1a762f159084081972fbf7a52e13266/users/bb7f084835fa4f3c954b615037b7ff22/roles/5d1feef0e13d4874928bd86258c4e3b2` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:54:55.619327 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be6b121-ab03-427a-bf06-acfcd96f3aec admin admin] RBAC: Authorizing `identity:create_grant(project_id=d1a762f159084081972fbf7a52e13266, user_id=bb7f084835fa4f3c954b615037b7ff22, role_id=5d1feef0e13d4874928bd86258c4e3b2)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:54:55.620354 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: WARNING py.warnings [None req-3be6b121-ab03-427a-bf06-acfcd96f3aec admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 01 14:54:55.620354 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: warnings.warn(msg) Sep 01 14:54:55.620354 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: Sep 01 14:54:55.622295 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be6b121-ab03-427a-bf06-acfcd96f3aec admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:54:55.645861 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 747/1501] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 1 14:54:55 2022] PUT /identity/v3/projects/d1a762f159084081972fbf7a52e13266/users/bb7f084835fa4f3c954b615037b7ff22/roles/5d1feef0e13d4874928bd86258c4e3b2 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:54:55.658466 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06029fc-e0fa-4d06-9f84-e54e70b8c9e3 None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:54:55.658853 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06029fc-e0fa-4d06-9f84-e54e70b8c9e3 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:54:55.659188 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06029fc-e0fa-4d06-9f84-e54e70b8c9e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:54:55.715800 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-c06029fc-e0fa-4d06-9f84-e54e70b8c9e3 None None] MFA Rules not processed for user `bb7f084835fa4f3c954b615037b7ff22`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:54:55.716929 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-c06029fc-e0fa-4d06-9f84-e54e70b8c9e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:54:55.791990 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 755/1502] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 14:54:55 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:54:55.805268 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7fb825-4f68-4d59-92eb-031bd5c7f391 None None] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:54:55.806096 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7fb825-4f68-4d59-92eb-031bd5c7f391 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:54:55.806361 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7fb825-4f68-4d59-92eb-031bd5c7f391 None None] PATH_INFO: `/` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:54:55.807685 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 748/1503] 10.222.0.33 () {54 vars in 964 bytes} [Thu Sep 1 14:54:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:54:55.812754 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ca9a91-1856-40be-9353-d7b68db454a0 None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:54:55.813038 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ca9a91-1856-40be-9353-d7b68db454a0 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:54:55.813311 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ca9a91-1856-40be-9353-d7b68db454a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:54:55.844359 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:54:55.890042 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-51ca9a91-1856-40be-9353-d7b68db454a0 None None] MFA Rules not processed for user `d3b81739e34e4e648f88b6d73fcb5e46`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:54:55.891267 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-51ca9a91-1856-40be-9353-d7b68db454a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:54:55.964268 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 756/1504] 10.222.0.33 () {58 vars in 1061 bytes} [Thu Sep 1 14:54:55 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:54:55.973757 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7fb825-4f68-4d59-92eb-031bd5c7f391 None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:54:55.975239 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-cd7fb825-4f68-4d59-92eb-031bd5c7f391 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:54:56.025396 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:54:56.080249 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7fb825-4f68-4d59-92eb-031bd5c7f391 None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:54:56.151994 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52acdb4-81e1-4c45-8d78-1e67f077c7f4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3b81739e34e4e648f88b6d73fcb5e46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:54:56.152592 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52acdb4-81e1-4c45-8d78-1e67f077c7f4 service cinder] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:54:56.152870 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52acdb4-81e1-4c45-8d78-1e67f077c7f4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:54:56.153138 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52acdb4-81e1-4c45-8d78-1e67f077c7f4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:54:56.154753 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-d52acdb4-81e1-4c45-8d78-1e67f077c7f4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:54:56.250804 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52acdb4-81e1-4c45-8d78-1e67f077c7f4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:54:56.252182 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52acdb4-81e1-4c45-8d78-1e67f077c7f4 service cinder] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:54:56.255606 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:54:56.269660 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 749/1505] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 14:54:55 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:54:56.314928 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d21910-aeae-4ab0-a393-a74be79cd0f8 None None] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:54:56.315218 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d21910-aeae-4ab0-a393-a74be79cd0f8 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:54:56.315494 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d21910-aeae-4ab0-a393-a74be79cd0f8 None None] PATH_INFO: `/` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:54:56.316766 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 757/1506] 10.222.0.33 () {52 vars in 830 bytes} [Thu Sep 1 14:54:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:54:56.320281 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52acdb4-81e1-4c45-8d78-1e67f077c7f4 service cinder] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:54:56.321519 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-d52acdb4-81e1-4c45-8d78-1e67f077c7f4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:54:56.418549 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52acdb4-81e1-4c45-8d78-1e67f077c7f4 service cinder] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:54:56.508736 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b1fe73-ed2e-45ad-b06d-3af16898410b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:54:56.509334 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b1fe73-ed2e-45ad-b06d-3af16898410b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:54:56.509584 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b1fe73-ed2e-45ad-b06d-3af16898410b admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:54:56.509852 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b1fe73-ed2e-45ad-b06d-3af16898410b admin admin] PATH_INFO: `/v3/users/bb7f084835fa4f3c954b615037b7ff22` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:54:56.521792 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b1fe73-ed2e-45ad-b06d-3af16898410b admin admin] RBAC: Authorizing `identity:delete_user(user_id=bb7f084835fa4f3c954b615037b7ff22)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:54:56.522838 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: WARNING py.warnings [None req-e6b1fe73-ed2e-45ad-b06d-3af16898410b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 01 14:54:56.522838 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: warnings.warn(msg) Sep 01 14:54:56.522838 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: Sep 01 14:54:56.524085 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b1fe73-ed2e-45ad-b06d-3af16898410b admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:54:56.534003 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:54:56.590081 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-e6b1fe73-ed2e-45ad-b06d-3af16898410b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bb7f084835fa4f3c954b615037b7ff22'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:54:56.599067 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-e6b1fe73-ed2e-45ad-b06d-3af16898410b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bb7f084835fa4f3c954b615037b7ff22'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:54:56.599707 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-e6b1fe73-ed2e-45ad-b06d-3af16898410b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bb7f084835fa4f3c954b615037b7ff22'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:54:56.623178 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-e6b1fe73-ed2e-45ad-b06d-3af16898410b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bb7f084835fa4f3c954b615037b7ff22'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:54:56.623629 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-e6b1fe73-ed2e-45ad-b06d-3af16898410b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bb7f084835fa4f3c954b615037b7ff22'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:54:56.642054 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 750/1507] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Sep 1 14:54:56 2022] DELETE /identity/v3/users/bb7f084835fa4f3c954b615037b7ff22 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:54:56.646387 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acac690-10ba-4f38-907c-429203f830df None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:54:56.646862 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acac690-10ba-4f38-907c-429203f830df None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:54:56.646967 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acac690-10ba-4f38-907c-429203f830df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:54:56.703636 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-1acac690-10ba-4f38-907c-429203f830df None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:54:56.704825 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-1acac690-10ba-4f38-907c-429203f830df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:54:56.780269 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:54:56.793330 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 758/1508] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 14:54:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:54:56.803348 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b1fe73-ed2e-45ad-b06d-3af16898410b admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:54:56.804567 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-e6b1fe73-ed2e-45ad-b06d-3af16898410b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:54:56.903070 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b1fe73-ed2e-45ad-b06d-3af16898410b admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:54:56.955818 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:54:56.977196 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1e51fb-a265-45b6-8547-0ae6b92d19ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2790149639aa418bbc4e4982e93eda44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:54:56.977842 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1e51fb-a265-45b6-8547-0ae6b92d19ce service neutron] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:54:56.978115 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1e51fb-a265-45b6-8547-0ae6b92d19ce service neutron] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:54:56.978414 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1e51fb-a265-45b6-8547-0ae6b92d19ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:54:56.979993 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-ca1e51fb-a265-45b6-8547-0ae6b92d19ce service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:54:57.088359 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1e51fb-a265-45b6-8547-0ae6b92d19ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:54:57.089800 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1e51fb-a265-45b6-8547-0ae6b92d19ce service neutron] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:54:57.109519 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 751/1509] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 14:54:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:54:57.425817 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acac690-10ba-4f38-907c-429203f830df None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:54:57.427113 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-1acac690-10ba-4f38-907c-429203f830df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:54:57.439113 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acac690-10ba-4f38-907c-429203f830df None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:54:57.536375 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491669ad-b4c1-4384-a19c-8b4a769f65f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:54:57.537133 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491669ad-b4c1-4384-a19c-8b4a769f65f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:54:57.537413 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491669ad-b4c1-4384-a19c-8b4a769f65f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:54:57.537704 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491669ad-b4c1-4384-a19c-8b4a769f65f0 admin admin] PATH_INFO: `/v3/projects/d1a762f159084081972fbf7a52e13266` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:54:57.548088 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491669ad-b4c1-4384-a19c-8b4a769f65f0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d1a762f159084081972fbf7a52e13266)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:54:57.549100 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: WARNING py.warnings [None req-491669ad-b4c1-4384-a19c-8b4a769f65f0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 01 14:54:57.549100 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: warnings.warn(msg) Sep 01 14:54:57.549100 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: Sep 01 14:54:57.550327 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491669ad-b4c1-4384-a19c-8b4a769f65f0 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:54:57.607685 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-491669ad-b4c1-4384-a19c-8b4a769f65f0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd1a762f159084081972fbf7a52e13266', 'request_id': 'req-491669ad-b4c1-4384-a19c-8b4a769f65f0'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:54:57.621477 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-491669ad-b4c1-4384-a19c-8b4a769f65f0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd1a762f159084081972fbf7a52e13266', 'request_id': 'req-491669ad-b4c1-4384-a19c-8b4a769f65f0'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:54:57.631426 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-491669ad-b4c1-4384-a19c-8b4a769f65f0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd1a762f159084081972fbf7a52e13266', 'request_id': 'req-491669ad-b4c1-4384-a19c-8b4a769f65f0'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:54:57.653606 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-491669ad-b4c1-4384-a19c-8b4a769f65f0 admin admin] The token cache is being invalidate because project d1a762f159084081972fbf7a52e13266 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81342) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:54:57.654207 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-491669ad-b4c1-4384-a19c-8b4a769f65f0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:54:57.654461 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-491669ad-b4c1-4384-a19c-8b4a769f65f0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:54:57.655800 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 759/1510] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Sep 1 14:54:57 2022] DELETE /identity/v3/projects/d1a762f159084081972fbf7a52e13266 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:55:07.107348 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=62 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:55:07.651514 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:56:27.501228 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b911ccc2-e86b-48f2-9435-4d8b91e87203 None None] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:56:27.501837 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b911ccc2-e86b-48f2-9435-4d8b91e87203 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:56:27.501837 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b911ccc2-e86b-48f2-9435-4d8b91e87203 None None] PATH_INFO: `/` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:56:27.503316 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 752/1511] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 1 14:56:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:56:27.507991 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef47c069-1fe4-45ca-af5e-a39d068e0463 None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:56:27.508397 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef47c069-1fe4-45ca-af5e-a39d068e0463 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:56:27.508540 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef47c069-1fe4-45ca-af5e-a39d068e0463 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:56:27.591015 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-ef47c069-1fe4-45ca-af5e-a39d068e0463 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:56:27.592239 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-ef47c069-1fe4-45ca-af5e-a39d068e0463 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:56:27.694697 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 760/1512] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 1 14:56:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:56:27.848467 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b911ccc2-e86b-48f2-9435-4d8b91e87203 None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:56:27.850586 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-b911ccc2-e86b-48f2-9435-4d8b91e87203 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:56:27.968393 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b911ccc2-e86b-48f2-9435-4d8b91e87203 None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:56:27.973717 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd27ebe3-f914-4e4c-9178-2dab8eb2923c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4858bb8621854ec8aba836e7d9050498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:56:27.974335 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd27ebe3-f914-4e4c-9178-2dab8eb2923c service nova] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:56:27.974613 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd27ebe3-f914-4e4c-9178-2dab8eb2923c service nova] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:56:27.974882 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd27ebe3-f914-4e4c-9178-2dab8eb2923c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:56:27.982693 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd27ebe3-f914-4e4c-9178-2dab8eb2923c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:56:27.984269 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd27ebe3-f914-4e4c-9178-2dab8eb2923c service nova] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:56:27.993407 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 753/1513] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 14:56:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:56:28.818669 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410a5266-d09d-4185-88f8-14f8abc9d86e None None] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:56:28.819198 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410a5266-d09d-4185-88f8-14f8abc9d86e None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:56:28.819198 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410a5266-d09d-4185-88f8-14f8abc9d86e None None] PATH_INFO: `/` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:56:28.820537 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 761/1514] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 1 14:56:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:56:28.825430 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3bfc99-64b2-4d7a-bb8e-fc87913e6462 None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:56:28.825910 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3bfc99-64b2-4d7a-bb8e-fc87913e6462 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:56:28.826024 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3bfc99-64b2-4d7a-bb8e-fc87913e6462 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:56:28.844884 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-7e3bfc99-64b2-4d7a-bb8e-fc87913e6462 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:56:28.846070 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-7e3bfc99-64b2-4d7a-bb8e-fc87913e6462 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:56:28.854213 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 754/1515] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 1 14:56:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:56:29.004233 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410a5266-d09d-4185-88f8-14f8abc9d86e None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:56:29.018226 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410a5266-d09d-4185-88f8-14f8abc9d86e None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:56:29.023590 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efee0ed-9772-44c7-931e-e16a1af0fe42 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4858bb8621854ec8aba836e7d9050498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:56:29.024179 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efee0ed-9772-44c7-931e-e16a1af0fe42 service nova] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:56:29.024954 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efee0ed-9772-44c7-931e-e16a1af0fe42 service nova] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:56:29.025220 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efee0ed-9772-44c7-931e-e16a1af0fe42 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:56:29.033283 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efee0ed-9772-44c7-931e-e16a1af0fe42 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:56:29.034917 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efee0ed-9772-44c7-931e-e16a1af0fe42 service nova] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:56:29.044890 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 762/1516] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 14:56:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:56:29.968484 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb8a997-1318-4e5a-bed7-5148bcdeae80 None None] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:56:29.969004 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb8a997-1318-4e5a-bed7-5148bcdeae80 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:56:29.969004 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb8a997-1318-4e5a-bed7-5148bcdeae80 None None] PATH_INFO: `/v3` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:56:29.970336 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 755/1517] 10.222.0.65 () {54 vars in 925 bytes} [Thu Sep 1 14:56:29 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:56:29.976486 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b6c3f5-36d9-4675-a38e-31e515beca1f None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:56:29.976935 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b6c3f5-36d9-4675-a38e-31e515beca1f None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:56:29.977049 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b6c3f5-36d9-4675-a38e-31e515beca1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:56:30.035051 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-40b6c3f5-36d9-4675-a38e-31e515beca1f None None] MFA Rules not processed for user `7f312c8dcabe48ef9ea453364ade38ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:56:30.036244 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-40b6c3f5-36d9-4675-a38e-31e515beca1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:56:30.052623 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:56:30.108726 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 763/1518] 10.222.0.65 () {58 vars in 1015 bytes} [Thu Sep 1 14:56:29 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:56:33.897979 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e78c74b-9953-47cd-a3e0-2993dd2ab087 None None] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:56:33.898521 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e78c74b-9953-47cd-a3e0-2993dd2ab087 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:56:33.898633 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e78c74b-9953-47cd-a3e0-2993dd2ab087 None None] PATH_INFO: `/` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:56:33.900473 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 756/1519] 10.222.0.33 () {54 vars in 961 bytes} [Thu Sep 1 14:56:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:56:33.905369 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af6f9a9-f000-4e90-a7d1-20c704e2c073 None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:56:33.905774 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af6f9a9-f000-4e90-a7d1-20c704e2c073 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:56:33.905906 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af6f9a9-f000-4e90-a7d1-20c704e2c073 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:56:33.924709 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-6af6f9a9-f000-4e90-a7d1-20c704e2c073 None None] MFA Rules not processed for user `7f312c8dcabe48ef9ea453364ade38ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:56:33.925894 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-6af6f9a9-f000-4e90-a7d1-20c704e2c073 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:56:33.934490 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 764/1520] 10.222.0.33 () {58 vars in 1058 bytes} [Thu Sep 1 14:56:33 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:56:33.941550 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e78c74b-9953-47cd-a3e0-2993dd2ab087 None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:56:33.955508 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e78c74b-9953-47cd-a3e0-2993dd2ab087 None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:56:33.960521 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f072f4-129a-44ee-a9ef-9e91e6cf1b71 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f312c8dcabe48ef9ea453364ade38ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:56:33.961130 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f072f4-129a-44ee-a9ef-9e91e6cf1b71 service placement] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:56:33.961409 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f072f4-129a-44ee-a9ef-9e91e6cf1b71 service placement] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:56:33.962315 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f072f4-129a-44ee-a9ef-9e91e6cf1b71 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:56:33.970423 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f072f4-129a-44ee-a9ef-9e91e6cf1b71 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:56:33.972005 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f072f4-129a-44ee-a9ef-9e91e6cf1b71 service placement] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:56:33.981432 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 757/1521] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 14:56:33 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 14:56:40.266074 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f1d80e-c0fa-43ea-94aa-127c140cf9e7 None None] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:56:40.266713 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f1d80e-c0fa-43ea-94aa-127c140cf9e7 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:56:40.266713 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f1d80e-c0fa-43ea-94aa-127c140cf9e7 None None] PATH_INFO: `/` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:56:40.268024 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 765/1522] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 1 14:56:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:56:40.272944 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c58dbf2-6450-49a9-ab10-ca397e599450 None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:56:40.273356 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c58dbf2-6450-49a9-ab10-ca397e599450 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:56:40.273483 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c58dbf2-6450-49a9-ab10-ca397e599450 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:56:40.293518 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-9c58dbf2-6450-49a9-ab10-ca397e599450 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:56:40.294764 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-9c58dbf2-6450-49a9-ab10-ca397e599450 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:56:40.302526 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 758/1523] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 1 14:56:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:56:40.457570 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f1d80e-c0fa-43ea-94aa-127c140cf9e7 None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:56:40.472831 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f1d80e-c0fa-43ea-94aa-127c140cf9e7 None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:56:40.479051 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af9957e-b790-4543-b259-875daee889b3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4858bb8621854ec8aba836e7d9050498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:56:40.479633 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af9957e-b790-4543-b259-875daee889b3 service nova] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:56:40.479905 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af9957e-b790-4543-b259-875daee889b3 service nova] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:56:40.480168 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af9957e-b790-4543-b259-875daee889b3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:56:40.487980 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af9957e-b790-4543-b259-875daee889b3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:56:40.489575 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af9957e-b790-4543-b259-875daee889b3 service nova] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:56:40.499385 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 766/1524] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 14:56:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:56:41.316133 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0010d2e-13f4-47d7-b976-d1608e25262b None None] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:56:41.316569 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0010d2e-13f4-47d7-b976-d1608e25262b None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:56:41.316695 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0010d2e-13f4-47d7-b976-d1608e25262b None None] PATH_INFO: `/` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:56:41.318012 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 759/1525] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 1 14:56:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:56:41.323561 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d27327e-e6a1-41a3-8061-96e34a5c52f7 None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:56:41.323995 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d27327e-e6a1-41a3-8061-96e34a5c52f7 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:56:41.324170 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d27327e-e6a1-41a3-8061-96e34a5c52f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:56:41.343084 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-6d27327e-e6a1-41a3-8061-96e34a5c52f7 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:56:41.344277 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-6d27327e-e6a1-41a3-8061-96e34a5c52f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:56:41.351952 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 767/1526] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 1 14:56:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:56:41.501078 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0010d2e-13f4-47d7-b976-d1608e25262b None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:56:41.515346 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0010d2e-13f4-47d7-b976-d1608e25262b None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:56:41.520722 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fce8eb6-478c-4d33-a01f-01a369ee3e65 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4858bb8621854ec8aba836e7d9050498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:56:41.521305 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fce8eb6-478c-4d33-a01f-01a369ee3e65 service nova] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:56:41.521583 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fce8eb6-478c-4d33-a01f-01a369ee3e65 service nova] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:56:41.521851 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fce8eb6-478c-4d33-a01f-01a369ee3e65 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:56:41.529825 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fce8eb6-478c-4d33-a01f-01a369ee3e65 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:56:41.531469 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fce8eb6-478c-4d33-a01f-01a369ee3e65 service nova] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:56:41.541217 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 760/1527] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 14:56:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:56:45.410725 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e649db9e-c3a1-4940-9788-170250090f43 None None] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:56:45.411249 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e649db9e-c3a1-4940-9788-170250090f43 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:56:45.411349 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e649db9e-c3a1-4940-9788-170250090f43 None None] PATH_INFO: `/` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:56:45.412617 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 768/1528] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 14:56:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:56:45.418307 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca4fe8c-6b29-42f0-9e1d-4047751f97f4 None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:56:45.418734 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca4fe8c-6b29-42f0-9e1d-4047751f97f4 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:56:45.418858 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca4fe8c-6b29-42f0-9e1d-4047751f97f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:56:45.438565 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-dca4fe8c-6b29-42f0-9e1d-4047751f97f4 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:56:45.439747 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-dca4fe8c-6b29-42f0-9e1d-4047751f97f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:56:45.447486 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 761/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 14:56:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:56:45.740743 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e649db9e-c3a1-4940-9788-170250090f43 None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:56:45.742517 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-e649db9e-c3a1-4940-9788-170250090f43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:56:45.823837 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e649db9e-c3a1-4940-9788-170250090f43 None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:56:45.827405 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda1fa13-2066-4812-93f9-871bca89669c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2790149639aa418bbc4e4982e93eda44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:56:45.828006 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda1fa13-2066-4812-93f9-871bca89669c service neutron] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:56:45.828283 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda1fa13-2066-4812-93f9-871bca89669c service neutron] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:56:45.828553 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda1fa13-2066-4812-93f9-871bca89669c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:56:45.837821 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda1fa13-2066-4812-93f9-871bca89669c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:56:45.839292 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda1fa13-2066-4812-93f9-871bca89669c service neutron] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:56:45.848533 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 769/1530] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 14:56:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:56:47.720127 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bf590f-388f-492a-8541-889f39eb8eb3 None None] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:56:47.720605 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bf590f-388f-492a-8541-889f39eb8eb3 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:56:47.720743 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bf590f-388f-492a-8541-889f39eb8eb3 None None] PATH_INFO: `/` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:56:47.721969 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 762/1531] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 14:56:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:56:47.727188 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ab9c8c-b8ad-4ea4-b2f7-237641e1bcd2 None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:56:47.727584 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ab9c8c-b8ad-4ea4-b2f7-237641e1bcd2 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:56:47.728290 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ab9c8c-b8ad-4ea4-b2f7-237641e1bcd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:56:47.747614 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-16ab9c8c-b8ad-4ea4-b2f7-237641e1bcd2 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:56:47.748811 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-16ab9c8c-b8ad-4ea4-b2f7-237641e1bcd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:56:47.756517 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 770/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 14:56:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:56:48.046278 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bf590f-388f-492a-8541-889f39eb8eb3 None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:56:48.057751 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bf590f-388f-492a-8541-889f39eb8eb3 None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:56:48.061249 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d6be46-3d2c-4efa-941f-5b2d53be5d0b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2790149639aa418bbc4e4982e93eda44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:56:48.061839 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d6be46-3d2c-4efa-941f-5b2d53be5d0b service neutron] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:56:48.062120 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d6be46-3d2c-4efa-941f-5b2d53be5d0b service neutron] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:56:48.062424 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d6be46-3d2c-4efa-941f-5b2d53be5d0b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:56:48.071781 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d6be46-3d2c-4efa-941f-5b2d53be5d0b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:56:48.073453 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d6be46-3d2c-4efa-941f-5b2d53be5d0b service neutron] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:56:48.084064 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 763/1533] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 14:56:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:56:51.515585 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef8900a-d555-46fa-97d1-7b4736c47887 None None] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:56:51.516120 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef8900a-d555-46fa-97d1-7b4736c47887 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:56:51.516120 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef8900a-d555-46fa-97d1-7b4736c47887 None None] PATH_INFO: `/` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:56:51.517445 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 771/1534] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 14:56:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:56:51.522588 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b96b9a5-89d4-4a91-8c0a-f00ef48182f0 None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:56:51.523018 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b96b9a5-89d4-4a91-8c0a-f00ef48182f0 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:56:51.523126 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b96b9a5-89d4-4a91-8c0a-f00ef48182f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:56:51.542199 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-5b96b9a5-89d4-4a91-8c0a-f00ef48182f0 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:56:51.543353 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-5b96b9a5-89d4-4a91-8c0a-f00ef48182f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:56:51.551087 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 764/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 14:56:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:56:52.098485 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef8900a-d555-46fa-97d1-7b4736c47887 None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:56:52.100137 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-1ef8900a-d555-46fa-97d1-7b4736c47887 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:56:52.162809 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:56:52.176397 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef8900a-d555-46fa-97d1-7b4736c47887 None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:56:52.179918 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25980e31-519f-4909-92b5-479f46f1a725 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3b81739e34e4e648f88b6d73fcb5e46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:56:52.181106 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25980e31-519f-4909-92b5-479f46f1a725 service cinder] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:56:52.181379 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25980e31-519f-4909-92b5-479f46f1a725 service cinder] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:56:52.181643 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25980e31-519f-4909-92b5-479f46f1a725 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:56:52.190495 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25980e31-519f-4909-92b5-479f46f1a725 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:56:52.191917 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25980e31-519f-4909-92b5-479f46f1a725 service cinder] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:56:52.200972 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 772/1536] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 14:56:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:56:54.090761 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9e92b7-841b-4818-bced-2c45948f2345 None None] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:56:54.091391 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9e92b7-841b-4818-bced-2c45948f2345 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:56:54.091391 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9e92b7-841b-4818-bced-2c45948f2345 None None] PATH_INFO: `/` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:56:54.092622 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 765/1537] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 14:56:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:56:54.097684 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3abbdfa-361d-4067-94a1-526388aa5412 None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:56:54.098116 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3abbdfa-361d-4067-94a1-526388aa5412 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:56:54.098226 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3abbdfa-361d-4067-94a1-526388aa5412 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:56:54.117426 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-e3abbdfa-361d-4067-94a1-526388aa5412 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:56:54.118663 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-e3abbdfa-361d-4067-94a1-526388aa5412 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:56:54.126591 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 773/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 14:56:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:56:54.672396 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9e92b7-841b-4818-bced-2c45948f2345 None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:56:54.674072 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-7e9e92b7-841b-4818-bced-2c45948f2345 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:56:54.687510 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9e92b7-841b-4818-bced-2c45948f2345 None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:56:54.690982 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f07fb9b-f165-4ddf-a289-e15bfe47be11 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3b81739e34e4e648f88b6d73fcb5e46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:56:54.691561 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f07fb9b-f165-4ddf-a289-e15bfe47be11 service cinder] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:56:54.691835 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f07fb9b-f165-4ddf-a289-e15bfe47be11 service cinder] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:56:54.692101 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f07fb9b-f165-4ddf-a289-e15bfe47be11 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:56:54.701342 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f07fb9b-f165-4ddf-a289-e15bfe47be11 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:56:54.702786 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f07fb9b-f165-4ddf-a289-e15bfe47be11 service cinder] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:56:54.711914 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 766/1539] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 14:56:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:57:04.112656 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=11 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:57:04.707837 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=89 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:58:18.191595 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c943b80-6466-4a07-ba8b-c13b1bb2dfd5 None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:18.192351 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c943b80-6466-4a07-ba8b-c13b1bb2dfd5 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:18.192351 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c943b80-6466-4a07-ba8b-c13b1bb2dfd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:18.220327 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-4c943b80-6466-4a07-ba8b-c13b1bb2dfd5 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:58:18.221779 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-4c943b80-6466-4a07-ba8b-c13b1bb2dfd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:58:18.234494 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 774/1540] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 14:58:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:58:18.244000 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953cad81-7ad7-4787-b335-9ebaca819976 None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:18.244635 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953cad81-7ad7-4787-b335-9ebaca819976 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:18.244635 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953cad81-7ad7-4787-b335-9ebaca819976 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:18.272456 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-953cad81-7ad7-4787-b335-9ebaca819976 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:58:18.273711 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-953cad81-7ad7-4787-b335-9ebaca819976 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:58:18.286410 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 767/1541] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 14:58:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:58:18.291293 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c943b80-6466-4a07-ba8b-c13b1bb2dfd5 None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:58:18.303494 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c943b80-6466-4a07-ba8b-c13b1bb2dfd5 None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:58:18.316014 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1466cf8d-42d8-4e66-a3b5-036c7ac97337 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:58:18.316635 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1466cf8d-42d8-4e66-a3b5-036c7ac97337 admin admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:18.316919 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1466cf8d-42d8-4e66-a3b5-036c7ac97337 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:18.317200 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1466cf8d-42d8-4e66-a3b5-036c7ac97337 admin admin] PATH_INFO: `/v3/domains` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:18.317904 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1466cf8d-42d8-4e66-a3b5-036c7ac97337 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:58:18.318209 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1466cf8d-42d8-4e66-a3b5-036c7ac97337 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:58:18.319849 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1466cf8d-42d8-4e66-a3b5-036c7ac97337 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:58:18.333062 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 775/1542] 10.222.0.33 () {58 vars in 1153 bytes} [Thu Sep 1 14:58:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:58:18.341515 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953cad81-7ad7-4787-b335-9ebaca819976 None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:58:18.351931 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953cad81-7ad7-4787-b335-9ebaca819976 None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:58:18.366766 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed8eb30-2f97-450c-80da-04f99813fce3 None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:18.366985 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed44915-f02b-4060-9496-4f4d9b4e633e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:58:18.367151 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed8eb30-2f97-450c-80da-04f99813fce3 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:18.367396 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed8eb30-2f97-450c-80da-04f99813fce3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:18.367519 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed44915-f02b-4060-9496-4f4d9b4e633e admin admin] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:18.367716 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed44915-f02b-4060-9496-4f4d9b4e633e admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:18.367990 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed44915-f02b-4060-9496-4f4d9b4e633e admin admin] PATH_INFO: `/v3/projects` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:18.368973 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed44915-f02b-4060-9496-4f4d9b4e633e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:58:18.371063 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed44915-f02b-4060-9496-4f4d9b4e633e admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:58:18.399309 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-1ed8eb30-2f97-450c-80da-04f99813fce3 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:58:18.401046 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-1ed8eb30-2f97-450c-80da-04f99813fce3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:58:18.416905 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 776/1543] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 14:58:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:58:18.425722 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099a34b0-05eb-4c1a-afe7-9e6a71c978d9 None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:18.426369 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099a34b0-05eb-4c1a-afe7-9e6a71c978d9 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:18.426931 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099a34b0-05eb-4c1a-afe7-9e6a71c978d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:18.442901 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 768/1544] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Sep 1 14:58:18 2022] POST /identity/v3/projects => generated 387 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:58:18.444173 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d8a6ca-08b6-4d8b-8174-9ab22e14b0d9 None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:18.444462 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d8a6ca-08b6-4d8b-8174-9ab22e14b0d9 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:18.444735 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d8a6ca-08b6-4d8b-8174-9ab22e14b0d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:18.459911 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-099a34b0-05eb-4c1a-afe7-9e6a71c978d9 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:58:18.461570 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-099a34b0-05eb-4c1a-afe7-9e6a71c978d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:58:18.482937 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-12d8a6ca-08b6-4d8b-8174-9ab22e14b0d9 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:58:18.484173 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-12d8a6ca-08b6-4d8b-8174-9ab22e14b0d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:58:18.567879 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 777/1545] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 14:58:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 14:58:18.568749 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099a34b0-05eb-4c1a-afe7-9e6a71c978d9 None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:58:18.569380 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 769/1546] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 14:58:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:58:18.570956 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc82b59-9181-4c51-8325-489d4c31cb81 None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:18.571239 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc82b59-9181-4c51-8325-489d4c31cb81 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:18.572957 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc82b59-9181-4c51-8325-489d4c31cb81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:18.591023 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099a34b0-05eb-4c1a-afe7-9e6a71c978d9 None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:58:18.597195 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3069e8d9-0b83-4f34-9a44-173f62f3352f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:58:18.597814 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3069e8d9-0b83-4f34-9a44-173f62f3352f admin admin] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:18.598091 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3069e8d9-0b83-4f34-9a44-173f62f3352f admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:18.598397 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3069e8d9-0b83-4f34-9a44-173f62f3352f admin admin] PATH_INFO: `/v3/users` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:18.599078 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3069e8d9-0b83-4f34-9a44-173f62f3352f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:58:18.601569 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3069e8d9-0b83-4f34-9a44-173f62f3352f admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:58:18.607642 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-1cc82b59-9181-4c51-8325-489d4c31cb81 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:58:18.609007 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-1cc82b59-9181-4c51-8325-489d4c31cb81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:58:18.619900 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 770/1547] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 14:58:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:58:18.621373 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc82b59-9181-4c51-8325-489d4c31cb81 None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:58:18.628410 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 778/1548] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Sep 1 14:58:18 2022] POST /identity/v3/users => generated 294 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:58:18.629771 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e59e801-8109-4272-acf1-ab544dc2bf86 None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:18.630356 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e59e801-8109-4272-acf1-ab544dc2bf86 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:18.630636 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e59e801-8109-4272-acf1-ab544dc2bf86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:18.632269 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc82b59-9181-4c51-8325-489d4c31cb81 None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:58:18.636854 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8bc64b-8426-4c19-abfc-80994934f919 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:58:18.637457 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8bc64b-8426-4c19-abfc-80994934f919 admin admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:18.637732 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8bc64b-8426-4c19-abfc-80994934f919 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:18.638038 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8bc64b-8426-4c19-abfc-80994934f919 admin admin] PATH_INFO: `/v3/domains` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:18.638843 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8bc64b-8426-4c19-abfc-80994934f919 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81341) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:58:18.639143 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8bc64b-8426-4c19-abfc-80994934f919 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:58:18.640197 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: WARNING py.warnings [None req-eb8bc64b-8426-4c19-abfc-80994934f919 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 01 14:58:18.640197 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: warnings.warn(msg) Sep 01 14:58:18.640197 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: Sep 01 14:58:18.641225 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8bc64b-8426-4c19-abfc-80994934f919 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:58:18.650772 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-2e59e801-8109-4272-acf1-ab544dc2bf86 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:58:18.651979 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-2e59e801-8109-4272-acf1-ab544dc2bf86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:58:18.654297 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 771/1549] 10.222.0.33 () {58 vars in 1153 bytes} [Thu Sep 1 14:58:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:58:18.655850 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0774d2-aab9-4b5e-9521-e48e788cefef None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:18.656133 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0774d2-aab9-4b5e-9521-e48e788cefef None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:18.656400 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0774d2-aab9-4b5e-9521-e48e788cefef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:18.665840 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 779/1550] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 14:58:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:58:18.667176 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa40e65-a9f8-4746-8964-d3f30c0cc2bd None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:18.667627 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa40e65-a9f8-4746-8964-d3f30c0cc2bd None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:18.667748 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa40e65-a9f8-4746-8964-d3f30c0cc2bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:18.681352 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-eb0774d2-aab9-4b5e-9521-e48e788cefef None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:58:18.682587 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-eb0774d2-aab9-4b5e-9521-e48e788cefef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:58:18.686993 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-4fa40e65-a9f8-4746-8964-d3f30c0cc2bd None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:58:18.688150 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-4fa40e65-a9f8-4746-8964-d3f30c0cc2bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:58:18.690825 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 772/1551] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 14:58:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:58:18.691936 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1a8e00-ff9e-478c-9536-6fac703aa412 None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:18.692217 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1a8e00-ff9e-478c-9536-6fac703aa412 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:18.692481 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1a8e00-ff9e-478c-9536-6fac703aa412 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:18.701565 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 780/1552] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 14:58:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:58:18.702996 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79907f42-7104-4077-bbc3-ef627887b043 None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:18.703280 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79907f42-7104-4077-bbc3-ef627887b043 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:18.703549 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79907f42-7104-4077-bbc3-ef627887b043 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:18.712776 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-3c1a8e00-ff9e-478c-9536-6fac703aa412 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:58:18.714405 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-3c1a8e00-ff9e-478c-9536-6fac703aa412 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:58:18.722211 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 773/1553] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 14:58:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:58:18.723085 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-79907f42-7104-4077-bbc3-ef627887b043 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:58:18.724914 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-79907f42-7104-4077-bbc3-ef627887b043 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:58:18.727091 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1a8e00-ff9e-478c-9536-6fac703aa412 None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:58:18.732885 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 781/1554] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 14:58:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:58:18.734097 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1318efa-42e4-4e47-8af6-d93309bd972e None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:18.734412 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1318efa-42e4-4e47-8af6-d93309bd972e None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:18.734684 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1318efa-42e4-4e47-8af6-d93309bd972e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:18.737197 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1a8e00-ff9e-478c-9536-6fac703aa412 None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:58:18.742687 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f9afdf-1459-4dc2-9ddf-6d96379e2235 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:58:18.743293 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f9afdf-1459-4dc2-9ddf-6d96379e2235 admin admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:18.743569 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f9afdf-1459-4dc2-9ddf-6d96379e2235 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:18.743837 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f9afdf-1459-4dc2-9ddf-6d96379e2235 admin admin] PATH_INFO: `/v3/roles` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:18.744479 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f9afdf-1459-4dc2-9ddf-6d96379e2235 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:58:18.746146 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f9afdf-1459-4dc2-9ddf-6d96379e2235 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:58:18.755322 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-d1318efa-42e4-4e47-8af6-d93309bd972e None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:58:18.756222 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 774/1555] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 1 14:58:18 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:58:18.756525 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-d1318efa-42e4-4e47-8af6-d93309bd972e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:58:18.756993 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f9afdf-1459-4dc2-9ddf-6d96379e2235 admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:58:18.764479 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 782/1556] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 14:58:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:58:18.765307 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1318efa-42e4-4e47-8af6-d93309bd972e None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:58:18.766371 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f9afdf-1459-4dc2-9ddf-6d96379e2235 admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:58:18.771587 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb877dee-278c-418c-bcbf-b72b33bcb3bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:58:18.772166 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb877dee-278c-418c-bcbf-b72b33bcb3bd admin admin] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:18.772442 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb877dee-278c-418c-bcbf-b72b33bcb3bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:18.772707 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb877dee-278c-418c-bcbf-b72b33bcb3bd admin admin] PATH_INFO: `/v3/projects` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:18.773321 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb877dee-278c-418c-bcbf-b72b33bcb3bd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:58:18.775246 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb877dee-278c-418c-bcbf-b72b33bcb3bd admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:58:18.775626 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1318efa-42e4-4e47-8af6-d93309bd972e None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:58:18.780011 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de78021-79cf-459f-9cb3-fd889414c896 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:58:18.780604 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de78021-79cf-459f-9cb3-fd889414c896 admin admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:18.780876 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de78021-79cf-459f-9cb3-fd889414c896 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:18.781139 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de78021-79cf-459f-9cb3-fd889414c896 admin admin] PATH_INFO: `/v3/domains` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:18.781772 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de78021-79cf-459f-9cb3-fd889414c896 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:58:18.782065 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de78021-79cf-459f-9cb3-fd889414c896 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:58:18.783689 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de78021-79cf-459f-9cb3-fd889414c896 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:58:18.797517 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 783/1557] 10.222.0.33 () {58 vars in 1153 bytes} [Thu Sep 1 14:58:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:58:18.798909 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828a2a43-62ec-4d17-b351-3a49594d269a None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:18.799117 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828a2a43-62ec-4d17-b351-3a49594d269a None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:18.799432 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828a2a43-62ec-4d17-b351-3a49594d269a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:18.807084 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 775/1558] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Sep 1 14:58:18 2022] POST /identity/v3/projects => generated 403 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:58:18.807084 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3713ef-a10b-443c-8405-1dc339d01eb4 None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:18.807084 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3713ef-a10b-443c-8405-1dc339d01eb4 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:18.807084 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3713ef-a10b-443c-8405-1dc339d01eb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:18.822400 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-828a2a43-62ec-4d17-b351-3a49594d269a None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:58:18.823590 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-828a2a43-62ec-4d17-b351-3a49594d269a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:58:18.825623 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-ac3713ef-a10b-443c-8405-1dc339d01eb4 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:58:18.826873 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-ac3713ef-a10b-443c-8405-1dc339d01eb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:58:18.878533 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:58:18.883408 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 784/1559] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 14:58:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:58:18.884643 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae47b573-4ee9-4eca-ab99-51b14b375c6d None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:18.885015 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae47b573-4ee9-4eca-ab99-51b14b375c6d None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:18.885212 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae47b573-4ee9-4eca-ab99-51b14b375c6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:18.891554 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 776/1560] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 14:58:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:58:18.892780 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bb0d4a-b8fc-4e8c-b69f-5d14129584a1 None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:18.893061 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bb0d4a-b8fc-4e8c-b69f-5d14129584a1 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:18.893331 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bb0d4a-b8fc-4e8c-b69f-5d14129584a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:18.908153 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-ae47b573-4ee9-4eca-ab99-51b14b375c6d None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:58:18.909354 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-ae47b573-4ee9-4eca-ab99-51b14b375c6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:58:18.917213 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-47bb0d4a-b8fc-4e8c-b69f-5d14129584a1 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:58:18.917355 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 785/1561] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 14:58:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:58:18.918178 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae47b573-4ee9-4eca-ab99-51b14b375c6d None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:58:18.918605 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-47bb0d4a-b8fc-4e8c-b69f-5d14129584a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:58:18.927151 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 777/1562] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 14:58:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:58:18.928073 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bb0d4a-b8fc-4e8c-b69f-5d14129584a1 None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:58:18.928330 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae47b573-4ee9-4eca-ab99-51b14b375c6d None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:58:18.933172 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133e9d15-bc01-40ec-9fd8-5da01e29fdd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:58:18.933794 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133e9d15-bc01-40ec-9fd8-5da01e29fdd1 admin admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:18.934069 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133e9d15-bc01-40ec-9fd8-5da01e29fdd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:18.934368 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133e9d15-bc01-40ec-9fd8-5da01e29fdd1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:18.935011 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133e9d15-bc01-40ec-9fd8-5da01e29fdd1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:58:18.936764 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133e9d15-bc01-40ec-9fd8-5da01e29fdd1 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:58:18.938021 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bb0d4a-b8fc-4e8c-b69f-5d14129584a1 None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:58:18.942066 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:58:18.942525 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374c507c-712d-413a-81a8-939048aa59c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:58:18.943613 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374c507c-712d-413a-81a8-939048aa59c1 admin admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:18.944019 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374c507c-712d-413a-81a8-939048aa59c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:18.944361 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374c507c-712d-413a-81a8-939048aa59c1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:18.945073 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374c507c-712d-413a-81a8-939048aa59c1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81341) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:58:18.945375 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374c507c-712d-413a-81a8-939048aa59c1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:58:18.947085 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374c507c-712d-413a-81a8-939048aa59c1 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:58:18.951196 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 786/1563] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 1 14:58:18 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:58:18.952064 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133e9d15-bc01-40ec-9fd8-5da01e29fdd1 admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:58:18.960711 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 778/1564] 10.222.0.33 () {58 vars in 1153 bytes} [Thu Sep 1 14:58:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:58:18.961536 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374c507c-712d-413a-81a8-939048aa59c1 admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:58:18.962031 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133e9d15-bc01-40ec-9fd8-5da01e29fdd1 admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:58:18.966815 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7f7e0e-ca15-4d11-8e59-345d8fc0ad57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:58:18.968533 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7f7e0e-ca15-4d11-8e59-345d8fc0ad57 admin admin] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:18.968807 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7f7e0e-ca15-4d11-8e59-345d8fc0ad57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:18.969069 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7f7e0e-ca15-4d11-8e59-345d8fc0ad57 admin admin] PATH_INFO: `/v3/projects` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:18.969681 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7f7e0e-ca15-4d11-8e59-345d8fc0ad57 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:58:18.970743 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: WARNING py.warnings [None req-ac7f7e0e-ca15-4d11-8e59-345d8fc0ad57 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 01 14:58:18.970743 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: warnings.warn(msg) Sep 01 14:58:18.970743 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: Sep 01 14:58:18.971925 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7f7e0e-ca15-4d11-8e59-345d8fc0ad57 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:58:18.972044 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374c507c-712d-413a-81a8-939048aa59c1 admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:58:18.976636 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1361b2-e7e5-4663-822d-306c576e07ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:58:18.977218 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1361b2-e7e5-4663-822d-306c576e07ac admin admin] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:18.977497 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1361b2-e7e5-4663-822d-306c576e07ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:18.977766 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1361b2-e7e5-4663-822d-306c576e07ac admin admin] PATH_INFO: `/v3/users` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:18.978448 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1361b2-e7e5-4663-822d-306c576e07ac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:58:18.979465 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: WARNING py.warnings [None req-be1361b2-e7e5-4663-822d-306c576e07ac admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 01 14:58:18.979465 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: warnings.warn(msg) Sep 01 14:58:18.979465 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: Sep 01 14:58:18.980663 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1361b2-e7e5-4663-822d-306c576e07ac admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:58:19.002892 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 787/1565] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Sep 1 14:58:18 2022] POST /identity/v3/projects => generated 377 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:58:19.002892 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7f7e0e-ca15-4d11-8e59-345d8fc0ad57 admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:58:19.003505 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 779/1566] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Sep 1 14:58:18 2022] POST /identity/v3/users => generated 302 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:58:19.003941 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1361b2-e7e5-4663-822d-306c576e07ac admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:58:19.014244 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1361b2-e7e5-4663-822d-306c576e07ac admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:58:19.065533 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7f7e0e-ca15-4d11-8e59-345d8fc0ad57 admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:58:19.070754 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622a6e81-8d99-4a94-901a-d8d50ade2a6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:58:19.071118 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccd835a-5b06-44b8-bd07-b3538fefa7a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:58:19.071361 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622a6e81-8d99-4a94-901a-d8d50ade2a6e admin admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:19.071640 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622a6e81-8d99-4a94-901a-d8d50ade2a6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:19.071764 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccd835a-5b06-44b8-bd07-b3538fefa7a5 admin admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:19.071914 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622a6e81-8d99-4a94-901a-d8d50ade2a6e admin admin] PATH_INFO: `/v3/domains` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:19.072018 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccd835a-5b06-44b8-bd07-b3538fefa7a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:19.072229 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccd835a-5b06-44b8-bd07-b3538fefa7a5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:19.072587 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622a6e81-8d99-4a94-901a-d8d50ade2a6e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:58:19.072888 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622a6e81-8d99-4a94-901a-d8d50ade2a6e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:58:19.072991 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eccd835a-5b06-44b8-bd07-b3538fefa7a5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81341) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:58:19.073182 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eccd835a-5b06-44b8-bd07-b3538fefa7a5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:58:19.074517 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622a6e81-8d99-4a94-901a-d8d50ade2a6e admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:58:19.074820 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eccd835a-5b06-44b8-bd07-b3538fefa7a5 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:58:19.087812 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 780/1567] 10.222.0.33 () {58 vars in 1153 bytes} [Thu Sep 1 14:58:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:58:19.094293 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 788/1568] 10.222.0.33 () {58 vars in 1153 bytes} [Thu Sep 1 14:58:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:58:19.094455 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccd835a-5b06-44b8-bd07-b3538fefa7a5 admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:58:19.094574 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622a6e81-8d99-4a94-901a-d8d50ade2a6e admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:58:19.105321 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccd835a-5b06-44b8-bd07-b3538fefa7a5 admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:58:19.106105 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622a6e81-8d99-4a94-901a-d8d50ade2a6e admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:58:19.109994 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eac82bf-1bb4-45d1-a521-9c9a0cf45026 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:58:19.110623 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eac82bf-1bb4-45d1-a521-9c9a0cf45026 admin admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:19.110815 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c22b9c-a7f3-4322-b2d0-de4f2033f084 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:58:19.110940 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eac82bf-1bb4-45d1-a521-9c9a0cf45026 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:19.111162 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eac82bf-1bb4-45d1-a521-9c9a0cf45026 admin admin] PATH_INFO: `/v3/domains` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:19.111407 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c22b9c-a7f3-4322-b2d0-de4f2033f084 admin admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:19.111687 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c22b9c-a7f3-4322-b2d0-de4f2033f084 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:19.111803 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eac82bf-1bb4-45d1-a521-9c9a0cf45026 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81341) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:58:19.111964 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c22b9c-a7f3-4322-b2d0-de4f2033f084 admin admin] PATH_INFO: `/v3/domains` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:19.112129 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eac82bf-1bb4-45d1-a521-9c9a0cf45026 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:58:19.112610 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c22b9c-a7f3-4322-b2d0-de4f2033f084 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:58:19.112972 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c22b9c-a7f3-4322-b2d0-de4f2033f084 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:58:19.113716 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eac82bf-1bb4-45d1-a521-9c9a0cf45026 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:58:19.114595 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c22b9c-a7f3-4322-b2d0-de4f2033f084 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:58:19.125970 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 781/1569] 10.222.0.33 () {58 vars in 1153 bytes} [Thu Sep 1 14:58:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:58:19.127066 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 789/1570] 10.222.0.33 () {58 vars in 1153 bytes} [Thu Sep 1 14:58:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:58:19.131652 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eac82bf-1bb4-45d1-a521-9c9a0cf45026 admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:58:19.131800 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c22b9c-a7f3-4322-b2d0-de4f2033f084 admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:58:19.141647 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eac82bf-1bb4-45d1-a521-9c9a0cf45026 admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:58:19.142059 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c22b9c-a7f3-4322-b2d0-de4f2033f084 admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:58:19.147076 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcbd055-bcfd-45e9-b76a-100a92649888 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:58:19.147240 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8d79d4-a6c3-4eb4-a7c2-58dbb8ae5111 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:58:19.147725 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcbd055-bcfd-45e9-b76a-100a92649888 admin admin] REQUEST_METHOD: `PUT` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:19.147863 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8d79d4-a6c3-4eb4-a7c2-58dbb8ae5111 admin admin] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:19.148044 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcbd055-bcfd-45e9-b76a-100a92649888 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:19.148161 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8d79d4-a6c3-4eb4-a7c2-58dbb8ae5111 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:19.148313 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcbd055-bcfd-45e9-b76a-100a92649888 admin admin] PATH_INFO: `/v3/projects/1d77f3fbed06493b8c31048594ec3020/users/440ce0ec02914568bfce9cd2649f3f6d/roles/5d1feef0e13d4874928bd86258c4e3b2` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:19.148411 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8d79d4-a6c3-4eb4-a7c2-58dbb8ae5111 admin admin] PATH_INFO: `/v3/projects` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:19.148899 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8d79d4-a6c3-4eb4-a7c2-58dbb8ae5111 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:58:19.150851 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8d79d4-a6c3-4eb4-a7c2-58dbb8ae5111 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:58:19.164231 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcbd055-bcfd-45e9-b76a-100a92649888 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d77f3fbed06493b8c31048594ec3020, user_id=440ce0ec02914568bfce9cd2649f3f6d, role_id=5d1feef0e13d4874928bd86258c4e3b2)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:58:19.166986 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcbd055-bcfd-45e9-b76a-100a92649888 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:58:19.177874 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 790/1571] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Sep 1 14:58:19 2022] POST /identity/v3/projects => generated 389 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:58:19.178707 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8d79d4-a6c3-4eb4-a7c2-58dbb8ae5111 admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:58:19.183701 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 782/1572] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 1 14:58:19 2022] PUT /identity/v3/projects/1d77f3fbed06493b8c31048594ec3020/users/440ce0ec02914568bfce9cd2649f3f6d/roles/5d1feef0e13d4874928bd86258c4e3b2 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:58:19.184495 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcbd055-bcfd-45e9-b76a-100a92649888 admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:58:19.244991 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcbd055-bcfd-45e9-b76a-100a92649888 admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:58:19.249625 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c50004-0012-4f37-9716-135ba4155f7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:58:19.250080 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8d79d4-a6c3-4eb4-a7c2-58dbb8ae5111 admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:58:19.250525 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c50004-0012-4f37-9716-135ba4155f7e admin admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:19.250525 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c50004-0012-4f37-9716-135ba4155f7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:19.250789 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c50004-0012-4f37-9716-135ba4155f7e admin admin] PATH_INFO: `/v3/roles` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:19.251429 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c50004-0012-4f37-9716-135ba4155f7e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:58:19.253044 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c50004-0012-4f37-9716-135ba4155f7e admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:58:19.254762 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a82fbe-2dbc-460c-8277-fac4f1716ba2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:58:19.255928 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a82fbe-2dbc-460c-8277-fac4f1716ba2 admin admin] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:19.256204 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a82fbe-2dbc-460c-8277-fac4f1716ba2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:19.256472 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a82fbe-2dbc-460c-8277-fac4f1716ba2 admin admin] PATH_INFO: `/v3/users` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:19.257100 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a82fbe-2dbc-460c-8277-fac4f1716ba2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:58:19.259041 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a82fbe-2dbc-460c-8277-fac4f1716ba2 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:58:19.263421 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 783/1573] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 1 14:58:19 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:58:19.264922 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c50004-0012-4f37-9716-135ba4155f7e admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:58:19.275811 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c50004-0012-4f37-9716-135ba4155f7e admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:58:19.283096 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 791/1574] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Sep 1 14:58:19 2022] POST /identity/v3/users => generated 289 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:58:19.283236 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c254850-fcbd-4e29-b3c7-881ada675a4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:58:19.283360 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a82fbe-2dbc-460c-8277-fac4f1716ba2 admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:58:19.283441 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c254850-fcbd-4e29-b3c7-881ada675a4c admin admin] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:19.283441 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c254850-fcbd-4e29-b3c7-881ada675a4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:19.283441 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c254850-fcbd-4e29-b3c7-881ada675a4c admin admin] PATH_INFO: `/v3/projects` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:19.283441 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c254850-fcbd-4e29-b3c7-881ada675a4c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:58:19.284233 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c254850-fcbd-4e29-b3c7-881ada675a4c admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:58:19.290196 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a82fbe-2dbc-460c-8277-fac4f1716ba2 admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:58:19.294792 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a2e728-1df7-4e52-9f58-55099fe88e42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:58:19.295368 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a2e728-1df7-4e52-9f58-55099fe88e42 admin admin] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:19.295646 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a2e728-1df7-4e52-9f58-55099fe88e42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:19.295916 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a2e728-1df7-4e52-9f58-55099fe88e42 admin admin] PATH_INFO: `/v3/projects` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:19.296055 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:58:19.296538 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a2e728-1df7-4e52-9f58-55099fe88e42 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:58:19.298508 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a2e728-1df7-4e52-9f58-55099fe88e42 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:58:19.312819 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 784/1575] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Sep 1 14:58:19 2022] POST /identity/v3/projects => generated 379 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:58:19.313690 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c254850-fcbd-4e29-b3c7-881ada675a4c admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:58:19.328753 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 792/1576] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Sep 1 14:58:19 2022] POST /identity/v3/projects => generated 375 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:58:19.329641 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a2e728-1df7-4e52-9f58-55099fe88e42 admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:58:19.334907 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:58:19.378030 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c254850-fcbd-4e29-b3c7-881ada675a4c admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:58:19.391220 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a2e728-1df7-4e52-9f58-55099fe88e42 admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:58:19.395814 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36f92dd-fdf3-4a54-b40a-9e33d47b2373 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:58:19.396440 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36f92dd-fdf3-4a54-b40a-9e33d47b2373 admin admin] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:19.396716 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36f92dd-fdf3-4a54-b40a-9e33d47b2373 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:19.396983 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36f92dd-fdf3-4a54-b40a-9e33d47b2373 admin admin] PATH_INFO: `/v3/projects` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:19.397603 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36f92dd-fdf3-4a54-b40a-9e33d47b2373 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:58:19.399537 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36f92dd-fdf3-4a54-b40a-9e33d47b2373 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:58:19.429921 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 793/1577] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Sep 1 14:58:19 2022] POST /identity/v3/projects => generated 393 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:58:19.429921 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36f92dd-fdf3-4a54-b40a-9e33d47b2373 admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:58:19.438081 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22da0fda-b060-4169-a2a0-4b8159914550 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:58:19.438715 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22da0fda-b060-4169-a2a0-4b8159914550 admin admin] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:19.438992 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22da0fda-b060-4169-a2a0-4b8159914550 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:19.439261 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22da0fda-b060-4169-a2a0-4b8159914550 admin admin] PATH_INFO: `/v3/projects` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:19.439877 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22da0fda-b060-4169-a2a0-4b8159914550 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:58:19.441761 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22da0fda-b060-4169-a2a0-4b8159914550 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:58:19.474185 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 785/1578] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Sep 1 14:58:19 2022] POST /identity/v3/projects => generated 373 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:58:19.474185 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8335bfbf-e8b2-47c3-bf2c-fb88f7e6b7d5 None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:19.474185 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8335bfbf-e8b2-47c3-bf2c-fb88f7e6b7d5 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:19.474185 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8335bfbf-e8b2-47c3-bf2c-fb88f7e6b7d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:19.493118 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36f92dd-fdf3-4a54-b40a-9e33d47b2373 admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:58:19.500781 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-8335bfbf-e8b2-47c3-bf2c-fb88f7e6b7d5 None None] MFA Rules not processed for user `440ce0ec02914568bfce9cd2649f3f6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:58:19.501931 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-8335bfbf-e8b2-47c3-bf2c-fb88f7e6b7d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:58:19.548367 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff887cdc-8763-48f6-9feb-fac67ec59be6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:58:19.548975 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff887cdc-8763-48f6-9feb-fac67ec59be6 admin admin] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:19.549246 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff887cdc-8763-48f6-9feb-fac67ec59be6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:19.549514 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff887cdc-8763-48f6-9feb-fac67ec59be6 admin admin] PATH_INFO: `/v3/users` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:19.550135 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff887cdc-8763-48f6-9feb-fac67ec59be6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:58:19.552270 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff887cdc-8763-48f6-9feb-fac67ec59be6 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:58:19.565986 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 786/1579] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 14:58:19 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:58:19.567258 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8335bfbf-e8b2-47c3-bf2c-fb88f7e6b7d5 None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:58:19.574934 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 794/1580] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Sep 1 14:58:19 2022] POST /identity/v3/users => generated 295 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:58:19.575779 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff887cdc-8763-48f6-9feb-fac67ec59be6 admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:58:19.578944 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8335bfbf-e8b2-47c3-bf2c-fb88f7e6b7d5 None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:58:19.583504 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78ebd5b-1d55-46c9-aa6b-2b6736539ebc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:58:19.584096 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78ebd5b-1d55-46c9-aa6b-2b6736539ebc admin admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:19.584379 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78ebd5b-1d55-46c9-aa6b-2b6736539ebc admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:19.584651 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78ebd5b-1d55-46c9-aa6b-2b6736539ebc admin admin] PATH_INFO: `/v3/roles` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:19.585290 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78ebd5b-1d55-46c9-aa6b-2b6736539ebc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:58:19.586479 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff887cdc-8763-48f6-9feb-fac67ec59be6 admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:58:19.586966 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78ebd5b-1d55-46c9-aa6b-2b6736539ebc admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:58:19.591055 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4d53f8-3508-4a4a-a339-4e57e74526f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:58:19.591631 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4d53f8-3508-4a4a-a339-4e57e74526f0 admin admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:19.591906 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4d53f8-3508-4a4a-a339-4e57e74526f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:19.592506 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4d53f8-3508-4a4a-a339-4e57e74526f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:19.592793 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4d53f8-3508-4a4a-a339-4e57e74526f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:58:19.594380 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4d53f8-3508-4a4a-a339-4e57e74526f0 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:58:19.596930 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 787/1581] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 1 14:58:19 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:58:19.597694 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78ebd5b-1d55-46c9-aa6b-2b6736539ebc admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:58:19.603065 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:58:19.608231 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 795/1582] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 1 14:58:19 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:58:19.608231 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4d53f8-3508-4a4a-a339-4e57e74526f0 admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:58:19.611122 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78ebd5b-1d55-46c9-aa6b-2b6736539ebc admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:58:19.617007 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cbb2bb-f482-4d2c-ae8f-ee6606db836f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:58:19.617626 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cbb2bb-f482-4d2c-ae8f-ee6606db836f admin admin] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:19.617848 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4d53f8-3508-4a4a-a339-4e57e74526f0 admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:58:19.617960 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cbb2bb-f482-4d2c-ae8f-ee6606db836f admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:19.618155 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cbb2bb-f482-4d2c-ae8f-ee6606db836f admin admin] PATH_INFO: `/v3/users` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:19.618854 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cbb2bb-f482-4d2c-ae8f-ee6606db836f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:58:19.620903 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cbb2bb-f482-4d2c-ae8f-ee6606db836f admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:58:19.622871 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097f2d99-afd6-4444-9ee2-bb2bf47a8eaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:58:19.623577 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097f2d99-afd6-4444-9ee2-bb2bf47a8eaa admin admin] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:19.623855 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097f2d99-afd6-4444-9ee2-bb2bf47a8eaa admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:19.624122 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097f2d99-afd6-4444-9ee2-bb2bf47a8eaa admin admin] PATH_INFO: `/v3/users` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:19.624739 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097f2d99-afd6-4444-9ee2-bb2bf47a8eaa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:58:19.626699 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097f2d99-afd6-4444-9ee2-bb2bf47a8eaa admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:58:19.641648 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 788/1583] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Sep 1 14:58:19 2022] POST /identity/v3/users => generated 290 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:58:19.642602 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cbb2bb-f482-4d2c-ae8f-ee6606db836f admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:58:19.642788 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:58:19.652413 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cbb2bb-f482-4d2c-ae8f-ee6606db836f admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:58:19.653319 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 796/1584] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Sep 1 14:58:19 2022] POST /identity/v3/users => generated 288 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:58:19.654209 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097f2d99-afd6-4444-9ee2-bb2bf47a8eaa admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:58:19.659011 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76633a0d-4ceb-4d22-8ab1-178a963c24bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:58:19.659795 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76633a0d-4ceb-4d22-8ab1-178a963c24bc admin admin] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:19.660253 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76633a0d-4ceb-4d22-8ab1-178a963c24bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:19.660524 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76633a0d-4ceb-4d22-8ab1-178a963c24bc admin admin] PATH_INFO: `/v3/users` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:19.661151 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76633a0d-4ceb-4d22-8ab1-178a963c24bc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:58:19.663670 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76633a0d-4ceb-4d22-8ab1-178a963c24bc admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:58:19.667478 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097f2d99-afd6-4444-9ee2-bb2bf47a8eaa admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:58:19.672260 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e906caf1-a1ef-4e0f-a2e7-33110cc5c94e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:58:19.672861 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e906caf1-a1ef-4e0f-a2e7-33110cc5c94e admin admin] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:19.673486 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e906caf1-a1ef-4e0f-a2e7-33110cc5c94e admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:19.673486 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e906caf1-a1ef-4e0f-a2e7-33110cc5c94e admin admin] PATH_INFO: `/v3/users` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:19.674110 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e906caf1-a1ef-4e0f-a2e7-33110cc5c94e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:58:19.676430 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e906caf1-a1ef-4e0f-a2e7-33110cc5c94e admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:58:19.698299 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 789/1585] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Sep 1 14:58:19 2022] POST /identity/v3/users => generated 297 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:58:19.702268 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76633a0d-4ceb-4d22-8ab1-178a963c24bc admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:58:19.702407 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 797/1586] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Sep 1 14:58:19 2022] POST /identity/v3/users => generated 287 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:58:19.703120 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e906caf1-a1ef-4e0f-a2e7-33110cc5c94e admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:58:19.713738 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76633a0d-4ceb-4d22-8ab1-178a963c24bc admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:58:19.714914 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e906caf1-a1ef-4e0f-a2e7-33110cc5c94e admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:58:19.718406 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e445ed-5768-460a-b565-12cace8e8bd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:58:19.719000 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e445ed-5768-460a-b565-12cace8e8bd6 admin admin] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:19.719264 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e445ed-5768-460a-b565-12cace8e8bd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:19.719448 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9969a9e2-0561-4503-a8e6-34ed1fa8d3ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:58:19.719561 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e445ed-5768-460a-b565-12cace8e8bd6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:19.720062 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9969a9e2-0561-4503-a8e6-34ed1fa8d3ce admin admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:19.720175 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e445ed-5768-460a-b565-12cace8e8bd6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:58:19.720344 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9969a9e2-0561-4503-a8e6-34ed1fa8d3ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:19.720611 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9969a9e2-0561-4503-a8e6-34ed1fa8d3ce admin admin] PATH_INFO: `/v3/roles` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:19.721244 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9969a9e2-0561-4503-a8e6-34ed1fa8d3ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:58:19.722022 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e445ed-5768-460a-b565-12cace8e8bd6 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:58:19.722860 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9969a9e2-0561-4503-a8e6-34ed1fa8d3ce admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:58:19.733169 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 798/1587] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 1 14:58:19 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:58:19.733937 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9969a9e2-0561-4503-a8e6-34ed1fa8d3ce admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:58:19.744361 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9969a9e2-0561-4503-a8e6-34ed1fa8d3ce admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:58:19.749186 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246b0d5d-d29d-40aa-97c7-512a084991e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:58:19.749944 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246b0d5d-d29d-40aa-97c7-512a084991e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:19.750220 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246b0d5d-d29d-40aa-97c7-512a084991e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:19.753303 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 790/1588] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Sep 1 14:58:19 2022] POST /identity/v3/projects => generated 385 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:58:19.753457 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246b0d5d-d29d-40aa-97c7-512a084991e6 admin admin] PATH_INFO: `/v3/projects/addc36256453406597172eeacd3d911e/users/93495b44907140a2a6665ff26d483421/roles/5d1feef0e13d4874928bd86258c4e3b2` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:19.753559 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e445ed-5768-460a-b565-12cace8e8bd6 admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:58:19.766833 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246b0d5d-d29d-40aa-97c7-512a084991e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=addc36256453406597172eeacd3d911e, user_id=93495b44907140a2a6665ff26d483421, role_id=5d1feef0e13d4874928bd86258c4e3b2)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:58:19.767879 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: WARNING py.warnings [None req-246b0d5d-d29d-40aa-97c7-512a084991e6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 01 14:58:19.767879 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: warnings.warn(msg) Sep 01 14:58:19.767879 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: Sep 01 14:58:19.769806 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246b0d5d-d29d-40aa-97c7-512a084991e6 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:58:19.784610 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 799/1589] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 1 14:58:19 2022] PUT /identity/v3/projects/addc36256453406597172eeacd3d911e/users/93495b44907140a2a6665ff26d483421/roles/5d1feef0e13d4874928bd86258c4e3b2 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:58:19.785420 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246b0d5d-d29d-40aa-97c7-512a084991e6 admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:58:19.816083 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e445ed-5768-460a-b565-12cace8e8bd6 admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:58:19.855838 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246b0d5d-d29d-40aa-97c7-512a084991e6 admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:58:19.860499 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247474a5-78a4-4227-99a9-f54de5dfa44c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:58:19.861101 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247474a5-78a4-4227-99a9-f54de5dfa44c admin admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:19.861384 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247474a5-78a4-4227-99a9-f54de5dfa44c admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:19.861653 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247474a5-78a4-4227-99a9-f54de5dfa44c admin admin] PATH_INFO: `/v3/roles` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:19.862334 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247474a5-78a4-4227-99a9-f54de5dfa44c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:58:19.863943 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247474a5-78a4-4227-99a9-f54de5dfa44c admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:58:19.873743 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 800/1590] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 1 14:58:19 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:58:19.874642 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247474a5-78a4-4227-99a9-f54de5dfa44c admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:58:19.877250 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7797c7e3-920d-4075-b382-9e98ab334680 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:58:19.877864 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7797c7e3-920d-4075-b382-9e98ab334680 admin admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:19.878143 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7797c7e3-920d-4075-b382-9e98ab334680 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:19.878463 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7797c7e3-920d-4075-b382-9e98ab334680 admin admin] PATH_INFO: `/v3/roles` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:19.879108 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7797c7e3-920d-4075-b382-9e98ab334680 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:58:19.881253 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7797c7e3-920d-4075-b382-9e98ab334680 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:58:19.886178 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247474a5-78a4-4227-99a9-f54de5dfa44c admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:58:19.890735 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f609cb5-1936-4bac-9bef-e9ea94ac7ef3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:58:19.891315 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 791/1591] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 1 14:58:19 2022] GET /identity/v3/roles => generated 1420 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:58:19.891443 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f609cb5-1936-4bac-9bef-e9ea94ac7ef3 admin admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:19.891595 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f609cb5-1936-4bac-9bef-e9ea94ac7ef3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:19.891869 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f609cb5-1936-4bac-9bef-e9ea94ac7ef3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:19.891970 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7797c7e3-920d-4075-b382-9e98ab334680 admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:58:19.892467 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f609cb5-1936-4bac-9bef-e9ea94ac7ef3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:58:19.894043 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f609cb5-1936-4bac-9bef-e9ea94ac7ef3 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:58:19.901959 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7797c7e3-920d-4075-b382-9e98ab334680 admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:58:19.907714 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 801/1592] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 1 14:58:19 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:58:19.907714 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f609cb5-1936-4bac-9bef-e9ea94ac7ef3 admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:58:19.908675 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5b26e1-81e0-4c96-8fe3-4f5a7e86c46e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:58:19.909257 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5b26e1-81e0-4c96-8fe3-4f5a7e86c46e admin admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:19.909538 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5b26e1-81e0-4c96-8fe3-4f5a7e86c46e admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:19.909810 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5b26e1-81e0-4c96-8fe3-4f5a7e86c46e admin admin] PATH_INFO: `/v3/roles` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:19.910464 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5b26e1-81e0-4c96-8fe3-4f5a7e86c46e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:58:19.912126 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5b26e1-81e0-4c96-8fe3-4f5a7e86c46e admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:58:19.917298 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f609cb5-1936-4bac-9bef-e9ea94ac7ef3 admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:58:19.922021 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45aee2e-7509-43fa-8d6b-99e951b6d32b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:58:19.922164 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 792/1593] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 1 14:58:19 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:58:19.922644 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45aee2e-7509-43fa-8d6b-99e951b6d32b admin admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:19.922927 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45aee2e-7509-43fa-8d6b-99e951b6d32b admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:19.923045 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5b26e1-81e0-4c96-8fe3-4f5a7e86c46e admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:58:19.923231 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45aee2e-7509-43fa-8d6b-99e951b6d32b admin admin] PATH_INFO: `/v3/roles` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:19.923934 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45aee2e-7509-43fa-8d6b-99e951b6d32b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:58:19.925660 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45aee2e-7509-43fa-8d6b-99e951b6d32b admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:58:19.933031 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5b26e1-81e0-4c96-8fe3-4f5a7e86c46e admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:58:19.939852 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 802/1594] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 1 14:58:19 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:58:19.939852 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45aee2e-7509-43fa-8d6b-99e951b6d32b admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:58:19.940729 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355ac80b-6f81-4f3d-b0e8-d98daab849b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:58:19.941311 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355ac80b-6f81-4f3d-b0e8-d98daab849b1 admin admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:19.941611 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355ac80b-6f81-4f3d-b0e8-d98daab849b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:19.941880 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355ac80b-6f81-4f3d-b0e8-d98daab849b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:19.942535 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355ac80b-6f81-4f3d-b0e8-d98daab849b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:58:19.944699 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355ac80b-6f81-4f3d-b0e8-d98daab849b1 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:58:19.949878 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45aee2e-7509-43fa-8d6b-99e951b6d32b admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:58:19.954422 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4498c76-9064-4c75-ab6e-7c2e1f35a8ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:58:19.954996 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4498c76-9064-4c75-ab6e-7c2e1f35a8ce admin admin] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:19.955234 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 793/1595] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 1 14:58:19 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:58:19.955334 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4498c76-9064-4c75-ab6e-7c2e1f35a8ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:19.955530 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4498c76-9064-4c75-ab6e-7c2e1f35a8ce admin admin] PATH_INFO: `/v3/users` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:19.956146 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4498c76-9064-4c75-ab6e-7c2e1f35a8ce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:58:19.956602 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b67066b-c94b-40c2-8b99-1fc0648e4ddf None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:19.956884 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b67066b-c94b-40c2-8b99-1fc0648e4ddf None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:19.957205 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b67066b-c94b-40c2-8b99-1fc0648e4ddf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:19.958052 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4498c76-9064-4c75-ab6e-7c2e1f35a8ce admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:58:19.982840 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 803/1596] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Sep 1 14:58:19 2022] POST /identity/v3/users => generated 299 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:58:19.983788 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4498c76-9064-4c75-ab6e-7c2e1f35a8ce admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:58:19.989073 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-2b67066b-c94b-40c2-8b99-1fc0648e4ddf None None] MFA Rules not processed for user `93495b44907140a2a6665ff26d483421`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:58:19.990888 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-2b67066b-c94b-40c2-8b99-1fc0648e4ddf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:58:19.994530 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4498c76-9064-4c75-ab6e-7c2e1f35a8ce admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:58:19.999170 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2ef98b-7892-45a4-8747-2313e999483b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:58:19.999769 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2ef98b-7892-45a4-8747-2313e999483b admin admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:20.000046 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2ef98b-7892-45a4-8747-2313e999483b admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:20.000319 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2ef98b-7892-45a4-8747-2313e999483b admin admin] PATH_INFO: `/v3/roles` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:20.000984 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2ef98b-7892-45a4-8747-2313e999483b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:58:20.002654 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2ef98b-7892-45a4-8747-2313e999483b admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:58:20.003943 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:58:20.014818 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 804/1597] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 1 14:58:19 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:58:20.015607 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2ef98b-7892-45a4-8747-2313e999483b admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:58:20.024925 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2ef98b-7892-45a4-8747-2313e999483b admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:58:20.030128 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b132ead3-659f-4c33-aefb-f6c109ed9969 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:58:20.030817 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b132ead3-659f-4c33-aefb-f6c109ed9969 admin admin] REQUEST_METHOD: `PUT` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:20.031093 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b132ead3-659f-4c33-aefb-f6c109ed9969 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:20.031357 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b132ead3-659f-4c33-aefb-f6c109ed9969 admin admin] PATH_INFO: `/v3/projects/b0202aa985634c8b8b241ce016600b19/users/82255a5060674d77a554e34f89ab1bff/roles/5d1feef0e13d4874928bd86258c4e3b2` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:20.045852 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b132ead3-659f-4c33-aefb-f6c109ed9969 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b0202aa985634c8b8b241ce016600b19, user_id=82255a5060674d77a554e34f89ab1bff, role_id=5d1feef0e13d4874928bd86258c4e3b2)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:58:20.048615 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b132ead3-659f-4c33-aefb-f6c109ed9969 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:58:20.061134 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 794/1598] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 14:58:19 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:58:20.062125 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b67066b-c94b-40c2-8b99-1fc0648e4ddf None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:58:20.063063 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 805/1599] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 1 14:58:20 2022] PUT /identity/v3/projects/b0202aa985634c8b8b241ce016600b19/users/82255a5060674d77a554e34f89ab1bff/roles/5d1feef0e13d4874928bd86258c4e3b2 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:58:20.063690 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b132ead3-659f-4c33-aefb-f6c109ed9969 admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:58:20.073229 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b67066b-c94b-40c2-8b99-1fc0648e4ddf None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:58:20.094773 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:58:20.130167 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b132ead3-659f-4c33-aefb-f6c109ed9969 admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:58:20.133488 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0d734b-9d43-446b-b874-f94d065b7d95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:58:20.134094 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0d734b-9d43-446b-b874-f94d065b7d95 admin admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:20.134405 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0d734b-9d43-446b-b874-f94d065b7d95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:20.134672 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0d734b-9d43-446b-b874-f94d065b7d95 admin admin] PATH_INFO: `/v3/roles` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:20.135303 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0d734b-9d43-446b-b874-f94d065b7d95 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:58:20.135542 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ea66e2-24a7-4762-b6ea-9f56fd755add admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:58:20.136118 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ea66e2-24a7-4762-b6ea-9f56fd755add admin admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:20.136392 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ea66e2-24a7-4762-b6ea-9f56fd755add admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:20.136655 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ea66e2-24a7-4762-b6ea-9f56fd755add admin admin] PATH_INFO: `/v3/roles` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:20.136910 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0d734b-9d43-446b-b874-f94d065b7d95 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:58:20.137286 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ea66e2-24a7-4762-b6ea-9f56fd755add admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:58:20.138896 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ea66e2-24a7-4762-b6ea-9f56fd755add admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:58:20.149753 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 795/1600] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 1 14:58:20 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:58:20.149753 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0d734b-9d43-446b-b874-f94d065b7d95 admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:58:20.150624 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 806/1601] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 1 14:58:20 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:58:20.151304 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ea66e2-24a7-4762-b6ea-9f56fd755add admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:58:20.158659 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0d734b-9d43-446b-b874-f94d065b7d95 admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:58:20.160787 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ea66e2-24a7-4762-b6ea-9f56fd755add admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:58:20.163358 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93aa8ef8-7387-4f95-b0e6-9fef2b423b24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:58:20.163935 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93aa8ef8-7387-4f95-b0e6-9fef2b423b24 admin admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:20.164209 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93aa8ef8-7387-4f95-b0e6-9fef2b423b24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:20.164478 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93aa8ef8-7387-4f95-b0e6-9fef2b423b24 admin admin] PATH_INFO: `/v3/roles` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:20.165094 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93aa8ef8-7387-4f95-b0e6-9fef2b423b24 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:58:20.165448 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9feb36a4-9ee5-4079-b6c8-0e9a7ecc9228 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:58:20.166069 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9feb36a4-9ee5-4079-b6c8-0e9a7ecc9228 admin admin] REQUEST_METHOD: `PUT` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:20.166379 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9feb36a4-9ee5-4079-b6c8-0e9a7ecc9228 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:20.166656 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9feb36a4-9ee5-4079-b6c8-0e9a7ecc9228 admin admin] PATH_INFO: `/v3/projects/2f02a6da81c440f9b61e4f2f90f37048/users/229e076684dd4bbdacda2231b18f0875/roles/5d1feef0e13d4874928bd86258c4e3b2` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:20.166752 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93aa8ef8-7387-4f95-b0e6-9fef2b423b24 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:58:20.176625 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 796/1602] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 1 14:58:20 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:58:20.178109 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93aa8ef8-7387-4f95-b0e6-9fef2b423b24 admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:58:20.182627 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9feb36a4-9ee5-4079-b6c8-0e9a7ecc9228 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f02a6da81c440f9b61e4f2f90f37048, user_id=229e076684dd4bbdacda2231b18f0875, role_id=5d1feef0e13d4874928bd86258c4e3b2)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:58:20.185272 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9feb36a4-9ee5-4079-b6c8-0e9a7ecc9228 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:58:20.187671 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93aa8ef8-7387-4f95-b0e6-9fef2b423b24 admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:58:20.193950 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a420732f-de51-424c-9603-ca34b9eb51af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:58:20.194724 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a420732f-de51-424c-9603-ca34b9eb51af admin admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:20.195275 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a420732f-de51-424c-9603-ca34b9eb51af admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:20.195563 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a420732f-de51-424c-9603-ca34b9eb51af admin admin] PATH_INFO: `/v3/roles` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:20.196192 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a420732f-de51-424c-9603-ca34b9eb51af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:58:20.197849 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a420732f-de51-424c-9603-ca34b9eb51af admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:58:20.199332 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 807/1603] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 1 14:58:20 2022] PUT /identity/v3/projects/2f02a6da81c440f9b61e4f2f90f37048/users/229e076684dd4bbdacda2231b18f0875/roles/5d1feef0e13d4874928bd86258c4e3b2 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:58:20.200104 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9feb36a4-9ee5-4079-b6c8-0e9a7ecc9228 admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:58:20.208844 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 797/1604] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 1 14:58:20 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:58:20.210267 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a04857-c63e-4906-950d-a99cd1b1bd99 None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:20.210697 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a04857-c63e-4906-950d-a99cd1b1bd99 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:20.211155 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a04857-c63e-4906-950d-a99cd1b1bd99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:20.231191 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-e7a04857-c63e-4906-950d-a99cd1b1bd99 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:58:20.232392 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-e7a04857-c63e-4906-950d-a99cd1b1bd99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:58:20.262055 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9feb36a4-9ee5-4079-b6c8-0e9a7ecc9228 admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:58:20.266572 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17832d36-f9da-4342-b61d-5f4c8c2659cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:58:20.267218 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17832d36-f9da-4342-b61d-5f4c8c2659cf admin admin] REQUEST_METHOD: `PUT` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:20.267495 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17832d36-f9da-4342-b61d-5f4c8c2659cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:20.267764 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17832d36-f9da-4342-b61d-5f4c8c2659cf admin admin] PATH_INFO: `/v3/projects/d925def3f55b435bb5452a0b5e5daff5/users/bb7098b844d743f5bafeb7ba15b3fdd7/roles/5d1feef0e13d4874928bd86258c4e3b2` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:20.281537 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17832d36-f9da-4342-b61d-5f4c8c2659cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=d925def3f55b435bb5452a0b5e5daff5, user_id=bb7098b844d743f5bafeb7ba15b3fdd7, role_id=5d1feef0e13d4874928bd86258c4e3b2)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:58:20.284245 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17832d36-f9da-4342-b61d-5f4c8c2659cf admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:58:20.290742 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 798/1605] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 14:58:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:58:20.291602 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a04857-c63e-4906-950d-a99cd1b1bd99 None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:58:20.302038 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 808/1606] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 1 14:58:20 2022] PUT /identity/v3/projects/d925def3f55b435bb5452a0b5e5daff5/users/bb7098b844d743f5bafeb7ba15b3fdd7/roles/5d1feef0e13d4874928bd86258c4e3b2 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:58:20.302334 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a04857-c63e-4906-950d-a99cd1b1bd99 None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:58:20.303364 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17832d36-f9da-4342-b61d-5f4c8c2659cf admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:58:20.332269 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:58:20.360843 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3668a7f-da41-4198-90ef-cc13e435ccd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:58:20.361440 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3668a7f-da41-4198-90ef-cc13e435ccd1 admin admin] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:20.361715 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3668a7f-da41-4198-90ef-cc13e435ccd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:20.361986 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3668a7f-da41-4198-90ef-cc13e435ccd1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:20.362637 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3668a7f-da41-4198-90ef-cc13e435ccd1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:58:20.364107 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17832d36-f9da-4342-b61d-5f4c8c2659cf admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:58:20.364618 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3668a7f-da41-4198-90ef-cc13e435ccd1 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:58:20.369797 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe39c6b-91e2-4b04-a065-03db41f42760 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:58:20.370476 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe39c6b-91e2-4b04-a065-03db41f42760 admin admin] REQUEST_METHOD: `PUT` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:20.370753 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe39c6b-91e2-4b04-a065-03db41f42760 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:20.371024 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe39c6b-91e2-4b04-a065-03db41f42760 admin admin] PATH_INFO: `/v3/projects/54e5a154e23440ce95f4644b4e2ee73f/users/c57d26d885514890b8b9d73548e81325/roles/5d1feef0e13d4874928bd86258c4e3b2` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:20.388278 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe39c6b-91e2-4b04-a065-03db41f42760 admin admin] RBAC: Authorizing `identity:create_grant(project_id=54e5a154e23440ce95f4644b4e2ee73f, user_id=c57d26d885514890b8b9d73548e81325, role_id=5d1feef0e13d4874928bd86258c4e3b2)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:58:20.391018 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe39c6b-91e2-4b04-a065-03db41f42760 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:58:20.392387 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 799/1607] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Sep 1 14:58:20 2022] POST /identity/v3/projects => generated 405 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:58:20.395194 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3668a7f-da41-4198-90ef-cc13e435ccd1 admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:58:20.405387 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 809/1608] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 1 14:58:20 2022] PUT /identity/v3/projects/54e5a154e23440ce95f4644b4e2ee73f/users/c57d26d885514890b8b9d73548e81325/roles/5d1feef0e13d4874928bd86258c4e3b2 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:58:20.406399 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe39c6b-91e2-4b04-a065-03db41f42760 admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:58:20.412395 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:58:20.461720 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3668a7f-da41-4198-90ef-cc13e435ccd1 admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:58:20.470744 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe39c6b-91e2-4b04-a065-03db41f42760 admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:58:20.475944 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1630099f-f807-48a5-b42f-9586c7028aea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:58:20.476587 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1630099f-f807-48a5-b42f-9586c7028aea admin admin] REQUEST_METHOD: `PUT` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:20.476859 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1630099f-f807-48a5-b42f-9586c7028aea admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:20.477122 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1630099f-f807-48a5-b42f-9586c7028aea admin admin] PATH_INFO: `/v3/projects/9af049e9440a458e96c0bdab2c71b5e0/users/76b06e441b9f45f99a80bce50f890f00/roles/5d1feef0e13d4874928bd86258c4e3b2` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:20.491415 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1630099f-f807-48a5-b42f-9586c7028aea admin admin] RBAC: Authorizing `identity:create_grant(project_id=9af049e9440a458e96c0bdab2c71b5e0, user_id=76b06e441b9f45f99a80bce50f890f00, role_id=5d1feef0e13d4874928bd86258c4e3b2)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:58:20.494036 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1630099f-f807-48a5-b42f-9586c7028aea admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:58:20.506741 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 810/1609] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 1 14:58:20 2022] PUT /identity/v3/projects/9af049e9440a458e96c0bdab2c71b5e0/users/76b06e441b9f45f99a80bce50f890f00/roles/5d1feef0e13d4874928bd86258c4e3b2 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:58:20.517623 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1b8343-858c-446b-8a7d-6a189c3631bd None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:20.517623 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1b8343-858c-446b-8a7d-6a189c3631bd None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:20.517623 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1b8343-858c-446b-8a7d-6a189c3631bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:20.520547 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3982b6-297c-4650-848d-a2c25b0110a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:58:20.521216 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3982b6-297c-4650-848d-a2c25b0110a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:20.521493 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3982b6-297c-4650-848d-a2c25b0110a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:20.521789 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3982b6-297c-4650-848d-a2c25b0110a8 admin admin] PATH_INFO: `/v3/projects/05e111f88e784eba929741c3cd250e78/users/df75151c37f74b6da7428dc290a274ff/roles/5d1feef0e13d4874928bd86258c4e3b2` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:20.526587 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-1f1b8343-858c-446b-8a7d-6a189c3631bd None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:58:20.527818 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-1f1b8343-858c-446b-8a7d-6a189c3631bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:58:20.537737 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3982b6-297c-4650-848d-a2c25b0110a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=05e111f88e784eba929741c3cd250e78, user_id=df75151c37f74b6da7428dc290a274ff, role_id=5d1feef0e13d4874928bd86258c4e3b2)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:58:20.540538 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3982b6-297c-4650-848d-a2c25b0110a8 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:58:20.555081 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 800/1610] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 1 14:58:20 2022] PUT /identity/v3/projects/05e111f88e784eba929741c3cd250e78/users/df75151c37f74b6da7428dc290a274ff/roles/5d1feef0e13d4874928bd86258c4e3b2 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:58:20.555958 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3982b6-297c-4650-848d-a2c25b0110a8 admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:58:20.588389 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 811/1611] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 14:58:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:58:20.589293 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1b8343-858c-446b-8a7d-6a189c3631bd None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:58:20.607682 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:58:20.623849 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3982b6-297c-4650-848d-a2c25b0110a8 admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:58:20.628536 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530c713e-e6ab-4ef7-830c-57b8c30a56bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:58:20.629192 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530c713e-e6ab-4ef7-830c-57b8c30a56bd admin admin] REQUEST_METHOD: `PUT` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:20.629501 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530c713e-e6ab-4ef7-830c-57b8c30a56bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:20.629770 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530c713e-e6ab-4ef7-830c-57b8c30a56bd admin admin] PATH_INFO: `/v3/projects/ae1a3cdeba4546b2b30ecdaf48ccd8cb/users/df61e99ad24c4d398ad96980af6c2b57/roles/2a2adb769e55476b82e5014897e0a9c0` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:20.645006 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530c713e-e6ab-4ef7-830c-57b8c30a56bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae1a3cdeba4546b2b30ecdaf48ccd8cb, user_id=df61e99ad24c4d398ad96980af6c2b57, role_id=2a2adb769e55476b82e5014897e0a9c0)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:58:20.645527 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1b8343-858c-446b-8a7d-6a189c3631bd None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:58:20.648266 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530c713e-e6ab-4ef7-830c-57b8c30a56bd admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:58:20.664331 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 801/1612] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 1 14:58:20 2022] PUT /identity/v3/projects/ae1a3cdeba4546b2b30ecdaf48ccd8cb/users/df61e99ad24c4d398ad96980af6c2b57/roles/2a2adb769e55476b82e5014897e0a9c0 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:58:20.665582 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f2654c-4ae1-4cdd-b404-6870cd1a93c9 None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:20.665872 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f2654c-4ae1-4cdd-b404-6870cd1a93c9 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:20.666149 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f2654c-4ae1-4cdd-b404-6870cd1a93c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:20.684887 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-58f2654c-4ae1-4cdd-b404-6870cd1a93c9 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:58:20.686199 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-58f2654c-4ae1-4cdd-b404-6870cd1a93c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:58:20.692952 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:58:20.701567 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c1aff9-6763-42ab-a292-12ac66eff705 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2790149639aa418bbc4e4982e93eda44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:58:20.702201 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c1aff9-6763-42ab-a292-12ac66eff705 service neutron] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:20.702521 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c1aff9-6763-42ab-a292-12ac66eff705 service neutron] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:20.702816 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c1aff9-6763-42ab-a292-12ac66eff705 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:20.712421 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c1aff9-6763-42ab-a292-12ac66eff705 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:58:20.713888 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c1aff9-6763-42ab-a292-12ac66eff705 service neutron] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:58:20.724957 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 812/1613] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 14:58:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:58:20.726361 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c1aff9-6763-42ab-a292-12ac66eff705 service neutron] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:58:20.736144 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c1aff9-6763-42ab-a292-12ac66eff705 service neutron] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:58:20.747315 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 802/1614] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 14:58:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:58:20.748786 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c38c24-fbc7-4b19-8f09-f891376dec23 None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:20.749069 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c38c24-fbc7-4b19-8f09-f891376dec23 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:20.749523 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c38c24-fbc7-4b19-8f09-f891376dec23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:20.778600 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-25c38c24-fbc7-4b19-8f09-f891376dec23 None None] MFA Rules not processed for user `c57d26d885514890b8b9d73548e81325`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:58:20.779827 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-25c38c24-fbc7-4b19-8f09-f891376dec23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:58:20.803722 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb780be-1eb9-4bec-b7bf-101e777c399a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:58:20.804368 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb780be-1eb9-4bec-b7bf-101e777c399a admin admin] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:20.804904 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb780be-1eb9-4bec-b7bf-101e777c399a admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:20.805174 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb780be-1eb9-4bec-b7bf-101e777c399a admin admin] PATH_INFO: `/v3/users` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:20.805812 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb780be-1eb9-4bec-b7bf-101e777c399a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:58:20.807905 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb780be-1eb9-4bec-b7bf-101e777c399a admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:58:20.830760 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 813/1615] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Sep 1 14:58:20 2022] POST /identity/v3/users => generated 309 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:58:20.832007 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d4e9c8-3c52-4ffc-941f-ac1bcf6ebb5b None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:20.832290 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d4e9c8-3c52-4ffc-941f-ac1bcf6ebb5b None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:20.832562 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d4e9c8-3c52-4ffc-941f-ac1bcf6ebb5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:20.854529 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 803/1616] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 14:58:20 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:58:20.860471 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7fa2fc-d6b9-485d-bade-a814426668ee None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:20.860566 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7fa2fc-d6b9-485d-bade-a814426668ee None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:20.860928 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7fa2fc-d6b9-485d-bade-a814426668ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:20.870663 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-33d4e9c8-3c52-4ffc-941f-ac1bcf6ebb5b None None] MFA Rules not processed for user `76b06e441b9f45f99a80bce50f890f00`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:58:20.871993 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-33d4e9c8-3c52-4ffc-941f-ac1bcf6ebb5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:58:20.882229 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-7b7fa2fc-d6b9-485d-bade-a814426668ee None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:58:20.883456 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-7b7fa2fc-d6b9-485d-bade-a814426668ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:58:20.894784 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 804/1617] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 14:58:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:58:20.900030 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7fa2fc-d6b9-485d-bade-a814426668ee None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:58:20.966214 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 814/1618] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 14:58:20 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:58:20.967149 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d4e9c8-3c52-4ffc-941f-ac1bcf6ebb5b None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:58:20.972668 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7fa2fc-d6b9-485d-bade-a814426668ee None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:58:20.977517 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3b006e-31ce-45f0-8e3e-911576ebb1af service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2790149639aa418bbc4e4982e93eda44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:58:20.978049 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d4e9c8-3c52-4ffc-941f-ac1bcf6ebb5b None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:58:20.978198 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3b006e-31ce-45f0-8e3e-911576ebb1af service neutron] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:20.978451 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3b006e-31ce-45f0-8e3e-911576ebb1af service neutron] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:20.978723 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3b006e-31ce-45f0-8e3e-911576ebb1af service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:20.982935 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea1f42d-13a0-4d73-8b54-bcd693e9e696 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:58:20.983546 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea1f42d-13a0-4d73-8b54-bcd693e9e696 admin admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:20.983881 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea1f42d-13a0-4d73-8b54-bcd693e9e696 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:20.984157 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea1f42d-13a0-4d73-8b54-bcd693e9e696 admin admin] PATH_INFO: `/v3/roles` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:20.984878 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea1f42d-13a0-4d73-8b54-bcd693e9e696 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:58:20.986767 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea1f42d-13a0-4d73-8b54-bcd693e9e696 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:58:20.987108 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:58:20.990058 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3b006e-31ce-45f0-8e3e-911576ebb1af service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:58:20.992262 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3b006e-31ce-45f0-8e3e-911576ebb1af service neutron] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:58:20.997577 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 815/1619] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 1 14:58:20 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:58:20.998447 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea1f42d-13a0-4d73-8b54-bcd693e9e696 admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:58:21.004033 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 805/1620] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 14:58:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:58:21.004847 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3b006e-31ce-45f0-8e3e-911576ebb1af service neutron] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:58:21.014206 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3b006e-31ce-45f0-8e3e-911576ebb1af service neutron] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:58:21.014809 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea1f42d-13a0-4d73-8b54-bcd693e9e696 admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:58:21.019172 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb560d5-81b7-42fa-b9fa-7bb800a144cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2790149639aa418bbc4e4982e93eda44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:58:21.019993 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb560d5-81b7-42fa-b9fa-7bb800a144cf service neutron] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:21.020300 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb560d5-81b7-42fa-b9fa-7bb800a144cf service neutron] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:21.020511 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902353dd-6647-4f2a-a31e-add33db3c59b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:58:21.020673 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb560d5-81b7-42fa-b9fa-7bb800a144cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:21.021945 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902353dd-6647-4f2a-a31e-add33db3c59b admin admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:21.022995 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902353dd-6647-4f2a-a31e-add33db3c59b admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:21.023541 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902353dd-6647-4f2a-a31e-add33db3c59b admin admin] PATH_INFO: `/v3/roles` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:21.025284 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902353dd-6647-4f2a-a31e-add33db3c59b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:58:21.028148 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902353dd-6647-4f2a-a31e-add33db3c59b admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:58:21.033604 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb560d5-81b7-42fa-b9fa-7bb800a144cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:58:21.035149 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb560d5-81b7-42fa-b9fa-7bb800a144cf service neutron] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:58:21.043220 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 806/1621] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 1 14:58:21 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:58:21.044293 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902353dd-6647-4f2a-a31e-add33db3c59b admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:58:21.046936 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-902353dd-6647-4f2a-a31e-add33db3c59b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:58:21.047151 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 816/1622] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 14:58:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:58:21.048400 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe5a6b3-1334-4cbe-81b3-446a1716ed03 None None] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:21.048747 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe5a6b3-1334-4cbe-81b3-446a1716ed03 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:21.049071 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe5a6b3-1334-4cbe-81b3-446a1716ed03 None None] PATH_INFO: `/` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:21.050362 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 817/1623] 10.222.0.33 () {54 vars in 962 bytes} [Thu Sep 1 14:58:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:58:21.055555 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe5a6b3-1334-4cbe-81b3-446a1716ed03 None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:58:21.061238 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902353dd-6647-4f2a-a31e-add33db3c59b admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:58:21.065573 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff82eef-1dd5-4cf0-bd16-8907bcc8fc89 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2790149639aa418bbc4e4982e93eda44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:58:21.066174 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff82eef-1dd5-4cf0-bd16-8907bcc8fc89 service neutron] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:21.066493 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff82eef-1dd5-4cf0-bd16-8907bcc8fc89 service neutron] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:21.066760 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff82eef-1dd5-4cf0-bd16-8907bcc8fc89 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:21.071857 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe5a6b3-1334-4cbe-81b3-446a1716ed03 None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:58:21.077393 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff82eef-1dd5-4cf0-bd16-8907bcc8fc89 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:58:21.077521 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57693699-ec7e-4c02-b9c9-14330c165352 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:58:21.078174 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57693699-ec7e-4c02-b9c9-14330c165352 admin admin] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:21.078488 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57693699-ec7e-4c02-b9c9-14330c165352 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:21.078757 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57693699-ec7e-4c02-b9c9-14330c165352 admin admin] PATH_INFO: `/v3/projects` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:21.080119 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff82eef-1dd5-4cf0-bd16-8907bcc8fc89 service neutron] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:58:21.080247 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57693699-ec7e-4c02-b9c9-14330c165352 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:58:21.083435 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57693699-ec7e-4c02-b9c9-14330c165352 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:58:21.093248 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 807/1624] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 14:58:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:58:21.093916 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff82eef-1dd5-4cf0-bd16-8907bcc8fc89 service neutron] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:58:21.105509 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff82eef-1dd5-4cf0-bd16-8907bcc8fc89 service neutron] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:58:21.112673 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc29c6f-5dfe-46da-860f-34f20952a193 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:58:21.113337 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc29c6f-5dfe-46da-860f-34f20952a193 admin admin] REQUEST_METHOD: `PUT` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:21.113622 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc29c6f-5dfe-46da-860f-34f20952a193 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:21.113876 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc29c6f-5dfe-46da-860f-34f20952a193 admin admin] PATH_INFO: `/v3/projects/ae1a3cdeba4546b2b30ecdaf48ccd8cb/users/df61e99ad24c4d398ad96980af6c2b57/roles/2a2adb769e55476b82e5014897e0a9c0` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:21.116193 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc29c6f-5dfe-46da-860f-34f20952a193 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae1a3cdeba4546b2b30ecdaf48ccd8cb, user_id=df61e99ad24c4d398ad96980af6c2b57, role_id=2a2adb769e55476b82e5014897e0a9c0)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:58:21.117437 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 818/1625] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Sep 1 14:58:21 2022] POST /identity/v3/projects => generated 371 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:58:21.118996 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57693699-ec7e-4c02-b9c9-14330c165352 admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:58:21.119261 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc29c6f-5dfe-46da-860f-34f20952a193 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:58:21.135519 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 808/1626] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 1 14:58:21 2022] PUT /identity/v3/projects/ae1a3cdeba4546b2b30ecdaf48ccd8cb/users/df61e99ad24c4d398ad96980af6c2b57/roles/2a2adb769e55476b82e5014897e0a9c0 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:58:21.136604 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815353df-1807-459a-9311-1dbe97735033 None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:21.136604 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815353df-1807-459a-9311-1dbe97735033 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:21.136880 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815353df-1807-459a-9311-1dbe97735033 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:21.154221 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:58:21.189823 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-815353df-1807-459a-9311-1dbe97735033 None None] MFA Rules not processed for user `4858bb8621854ec8aba836e7d9050498`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:58:21.191559 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-815353df-1807-459a-9311-1dbe97735033 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:58:21.214221 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57693699-ec7e-4c02-b9c9-14330c165352 admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:58:21.288496 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e492657e-d428-4f5d-9b30-25c3a24b4da6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:58:21.289136 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e492657e-d428-4f5d-9b30-25c3a24b4da6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:21.289378 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e492657e-d428-4f5d-9b30-25c3a24b4da6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:21.289604 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e492657e-d428-4f5d-9b30-25c3a24b4da6 admin admin] PATH_INFO: `/v3/projects/4a4b43c191584f83a131556d29c0b0f7/users/8e4b3ada9f154e839231d5ae2904a42b/roles/2a2adb769e55476b82e5014897e0a9c0` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:21.291252 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 809/1627] 10.222.0.33 () {58 vars in 1059 bytes} [Thu Sep 1 14:58:21 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:58:21.292391 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815353df-1807-459a-9311-1dbe97735033 None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:58:21.307180 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815353df-1807-459a-9311-1dbe97735033 None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:58:21.308861 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e492657e-d428-4f5d-9b30-25c3a24b4da6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a4b43c191584f83a131556d29c0b0f7, user_id=8e4b3ada9f154e839231d5ae2904a42b, role_id=2a2adb769e55476b82e5014897e0a9c0)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:58:21.312051 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e492657e-d428-4f5d-9b30-25c3a24b4da6 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:58:21.315120 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18902ef-b5f6-468c-a8c4-fa6bf94c5349 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:58:21.316420 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18902ef-b5f6-468c-a8c4-fa6bf94c5349 admin admin] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:21.316915 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18902ef-b5f6-468c-a8c4-fa6bf94c5349 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:21.317254 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18902ef-b5f6-468c-a8c4-fa6bf94c5349 admin admin] PATH_INFO: `/v3/users` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:21.318700 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18902ef-b5f6-468c-a8c4-fa6bf94c5349 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:58:21.321056 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18902ef-b5f6-468c-a8c4-fa6bf94c5349 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:58:21.330010 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 819/1628] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 1 14:58:21 2022] PUT /identity/v3/projects/4a4b43c191584f83a131556d29c0b0f7/users/8e4b3ada9f154e839231d5ae2904a42b/roles/2a2adb769e55476b82e5014897e0a9c0 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:58:21.332438 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edae3a8a-4ea1-47ce-b673-0bb632e63602 None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:21.335429 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edae3a8a-4ea1-47ce-b673-0bb632e63602 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:21.335921 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edae3a8a-4ea1-47ce-b673-0bb632e63602 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:21.354263 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 810/1629] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Sep 1 14:58:21 2022] POST /identity/v3/users => generated 292 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:58:21.355716 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18902ef-b5f6-468c-a8c4-fa6bf94c5349 admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:58:21.368377 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-edae3a8a-4ea1-47ce-b673-0bb632e63602 None None] MFA Rules not processed for user `df61e99ad24c4d398ad96980af6c2b57`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:58:21.370003 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-edae3a8a-4ea1-47ce-b673-0bb632e63602 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:58:21.374670 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18902ef-b5f6-468c-a8c4-fa6bf94c5349 admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:58:21.458880 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 820/1630] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 14:58:21 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:58:21.460100 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edae3a8a-4ea1-47ce-b673-0bb632e63602 None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:58:21.473975 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba7c3a5-1056-464f-bb69-4ab51682cd1b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4858bb8621854ec8aba836e7d9050498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:58:21.475106 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba7c3a5-1056-464f-bb69-4ab51682cd1b service nova] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:21.475424 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba7c3a5-1056-464f-bb69-4ab51682cd1b service nova] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:21.475698 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba7c3a5-1056-464f-bb69-4ab51682cd1b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:21.494693 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:58:21.506009 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:58:21.536646 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edae3a8a-4ea1-47ce-b673-0bb632e63602 None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:58:21.543301 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31e892d-023d-48fe-95c6-67bffea99960 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:58:21.544174 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31e892d-023d-48fe-95c6-67bffea99960 admin admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:21.544456 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31e892d-023d-48fe-95c6-67bffea99960 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:21.544732 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31e892d-023d-48fe-95c6-67bffea99960 admin admin] PATH_INFO: `/v3/roles` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:21.545558 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31e892d-023d-48fe-95c6-67bffea99960 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:58:21.547864 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31e892d-023d-48fe-95c6-67bffea99960 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:58:21.557182 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba7c3a5-1056-464f-bb69-4ab51682cd1b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:58:21.562064 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba7c3a5-1056-464f-bb69-4ab51682cd1b service nova] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:58:21.564676 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 821/1631] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 1 14:58:21 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:58:21.565488 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31e892d-023d-48fe-95c6-67bffea99960 admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:58:21.580382 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31e892d-023d-48fe-95c6-67bffea99960 admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:58:21.585666 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 811/1632] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 14:58:21 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 231 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:58:21.586185 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b68cd6c-2a8e-47f2-b2f9-812ba85ea806 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:58:21.586822 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b68cd6c-2a8e-47f2-b2f9-812ba85ea806 admin admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:21.586982 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba7c3a5-1056-464f-bb69-4ab51682cd1b service nova] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:58:21.587096 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b68cd6c-2a8e-47f2-b2f9-812ba85ea806 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:21.587356 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b68cd6c-2a8e-47f2-b2f9-812ba85ea806 admin admin] PATH_INFO: `/v3/roles` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:21.588058 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b68cd6c-2a8e-47f2-b2f9-812ba85ea806 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:58:21.589715 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b68cd6c-2a8e-47f2-b2f9-812ba85ea806 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:58:21.605150 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 822/1633] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 1 14:58:21 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:58:21.605150 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b68cd6c-2a8e-47f2-b2f9-812ba85ea806 admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:58:21.632556 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b68cd6c-2a8e-47f2-b2f9-812ba85ea806 admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:58:21.649200 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0542a900-c421-4521-b032-af807533bf54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:58:21.655887 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0542a900-c421-4521-b032-af807533bf54 admin admin] REQUEST_METHOD: `PUT` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:21.656174 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0542a900-c421-4521-b032-af807533bf54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:21.662868 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0542a900-c421-4521-b032-af807533bf54 admin admin] PATH_INFO: `/v3/projects/4a4b43c191584f83a131556d29c0b0f7/users/8e4b3ada9f154e839231d5ae2904a42b/roles/2a2adb769e55476b82e5014897e0a9c0` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:21.667052 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0542a900-c421-4521-b032-af807533bf54 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a4b43c191584f83a131556d29c0b0f7, user_id=8e4b3ada9f154e839231d5ae2904a42b, role_id=2a2adb769e55476b82e5014897e0a9c0)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:58:21.680080 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0542a900-c421-4521-b032-af807533bf54 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:58:21.716353 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 823/1634] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 1 14:58:21 2022] PUT /identity/v3/projects/4a4b43c191584f83a131556d29c0b0f7/users/8e4b3ada9f154e839231d5ae2904a42b/roles/2a2adb769e55476b82e5014897e0a9c0 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:58:21.717913 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0542a900-c421-4521-b032-af807533bf54 admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:58:21.734652 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba7c3a5-1056-464f-bb69-4ab51682cd1b service nova] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:58:21.868873 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d598aa7e-3c12-41f4-b0a8-19a47806b515 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3b81739e34e4e648f88b6d73fcb5e46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:58:21.870573 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d598aa7e-3c12-41f4-b0a8-19a47806b515 service cinder] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:21.874036 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d598aa7e-3c12-41f4-b0a8-19a47806b515 service cinder] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:21.874358 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d598aa7e-3c12-41f4-b0a8-19a47806b515 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:21.888630 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0542a900-c421-4521-b032-af807533bf54 admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:58:21.901785 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d17ca8-9e0b-42ff-a9c3-045e349e5d4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:58:21.903971 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d17ca8-9e0b-42ff-a9c3-045e349e5d4f admin admin] REQUEST_METHOD: `PUT` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:21.905145 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d17ca8-9e0b-42ff-a9c3-045e349e5d4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:21.905436 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d17ca8-9e0b-42ff-a9c3-045e349e5d4f admin admin] PATH_INFO: `/v3/projects/2e452af76635415ebc7b06a4b9f625e6/users/8313040cb0214c06979985a9a5f6e5ab/roles/2a2adb769e55476b82e5014897e0a9c0` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:21.958142 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d17ca8-9e0b-42ff-a9c3-045e349e5d4f admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e452af76635415ebc7b06a4b9f625e6, user_id=8313040cb0214c06979985a9a5f6e5ab, role_id=2a2adb769e55476b82e5014897e0a9c0)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:58:21.968061 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d17ca8-9e0b-42ff-a9c3-045e349e5d4f admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:58:22.009761 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 824/1635] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 1 14:58:21 2022] PUT /identity/v3/projects/2e452af76635415ebc7b06a4b9f625e6/users/8313040cb0214c06979985a9a5f6e5ab/roles/2a2adb769e55476b82e5014897e0a9c0 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:58:22.009761 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f32975-b215-4d9f-9ca4-02c5f733f056 None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:22.009761 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f32975-b215-4d9f-9ca4-02c5f733f056 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:22.010145 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f32975-b215-4d9f-9ca4-02c5f733f056 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:22.052992 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d598aa7e-3c12-41f4-b0a8-19a47806b515 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:58:22.058945 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d598aa7e-3c12-41f4-b0a8-19a47806b515 service cinder] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:58:22.059232 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-90f32975-b215-4d9f-9ca4-02c5f733f056 None None] MFA Rules not processed for user `8e4b3ada9f154e839231d5ae2904a42b`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:58:22.059232 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-90f32975-b215-4d9f-9ca4-02c5f733f056 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:58:22.094829 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 812/1636] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 14:58:21 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 508 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:58:22.095912 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d598aa7e-3c12-41f4-b0a8-19a47806b515 service cinder] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:58:22.119433 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d598aa7e-3c12-41f4-b0a8-19a47806b515 service cinder] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:58:22.164749 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:58:22.192245 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 825/1637] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 14:58:22 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:58:22.229012 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f32975-b215-4d9f-9ca4-02c5f733f056 None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:58:22.254943 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338dc077-3388-4e74-8837-6bc42d03c3ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:58:22.256898 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338dc077-3388-4e74-8837-6bc42d03c3ff admin admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:22.257246 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338dc077-3388-4e74-8837-6bc42d03c3ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:22.257589 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338dc077-3388-4e74-8837-6bc42d03c3ff admin admin] PATH_INFO: `/v3/roles` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:22.258329 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338dc077-3388-4e74-8837-6bc42d03c3ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:58:22.260075 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338dc077-3388-4e74-8837-6bc42d03c3ff admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:58:22.266360 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:58:22.276535 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 813/1638] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 1 14:58:22 2022] GET /identity/v3/roles => generated 1420 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:58:22.277869 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338dc077-3388-4e74-8837-6bc42d03c3ff admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:58:22.290656 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338dc077-3388-4e74-8837-6bc42d03c3ff admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:58:22.303137 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0876eb-b71e-48d2-9200-ab99382922d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:58:22.305423 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0876eb-b71e-48d2-9200-ab99382922d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:22.305771 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0876eb-b71e-48d2-9200-ab99382922d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:22.308654 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0876eb-b71e-48d2-9200-ab99382922d7 admin admin] PATH_INFO: `/v3/users/440ce0ec02914568bfce9cd2649f3f6d` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:22.311100 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0876eb-b71e-48d2-9200-ab99382922d7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=440ce0ec02914568bfce9cd2649f3f6d)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:58:22.314177 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0876eb-b71e-48d2-9200-ab99382922d7 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:58:22.375427 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f32975-b215-4d9f-9ca4-02c5f733f056 None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:58:22.389687 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8623fc1e-6e6f-41c9-af22-3f6a9b433f53 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3b81739e34e4e648f88b6d73fcb5e46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:58:22.389932 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-7f0876eb-b71e-48d2-9200-ab99382922d7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '440ce0ec02914568bfce9cd2649f3f6d'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:58:22.390343 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8623fc1e-6e6f-41c9-af22-3f6a9b433f53 service cinder] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:22.390631 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8623fc1e-6e6f-41c9-af22-3f6a9b433f53 service cinder] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:22.390906 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8623fc1e-6e6f-41c9-af22-3f6a9b433f53 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:22.404688 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8623fc1e-6e6f-41c9-af22-3f6a9b433f53 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:58:22.406775 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8623fc1e-6e6f-41c9-af22-3f6a9b433f53 service cinder] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:58:22.414798 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-7f0876eb-b71e-48d2-9200-ab99382922d7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '440ce0ec02914568bfce9cd2649f3f6d'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:58:22.415875 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-7f0876eb-b71e-48d2-9200-ab99382922d7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '440ce0ec02914568bfce9cd2649f3f6d'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:58:22.432673 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 826/1639] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 14:58:22 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:58:22.433499 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8623fc1e-6e6f-41c9-af22-3f6a9b433f53 service cinder] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:58:22.438630 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-7f0876eb-b71e-48d2-9200-ab99382922d7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '440ce0ec02914568bfce9cd2649f3f6d'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:58:22.438630 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-7f0876eb-b71e-48d2-9200-ab99382922d7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '440ce0ec02914568bfce9cd2649f3f6d'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:58:22.457499 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8623fc1e-6e6f-41c9-af22-3f6a9b433f53 service cinder] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:58:22.463557 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f7c15d-93c4-4a1b-9236-e8338d58b111 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:58:22.464210 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f7c15d-93c4-4a1b-9236-e8338d58b111 admin admin] REQUEST_METHOD: `PUT` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:22.464491 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f7c15d-93c4-4a1b-9236-e8338d58b111 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:22.464763 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f7c15d-93c4-4a1b-9236-e8338d58b111 admin admin] PATH_INFO: `/v3/projects/2e452af76635415ebc7b06a4b9f625e6/users/8313040cb0214c06979985a9a5f6e5ab/roles/2a2adb769e55476b82e5014897e0a9c0` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:22.469372 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f7c15d-93c4-4a1b-9236-e8338d58b111 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e452af76635415ebc7b06a4b9f625e6, user_id=8313040cb0214c06979985a9a5f6e5ab, role_id=2a2adb769e55476b82e5014897e0a9c0)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:58:22.470619 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f7c15d-93c4-4a1b-9236-e8338d58b111 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:58:22.487417 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 814/1640] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Sep 1 14:58:22 2022] DELETE /identity/v3/users/440ce0ec02914568bfce9cd2649f3f6d => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:58:22.487417 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576e0643-4cd8-4b30-a6b0-be49fca5055f None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:22.487417 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576e0643-4cd8-4b30-a6b0-be49fca5055f None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:22.487417 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576e0643-4cd8-4b30-a6b0-be49fca5055f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:22.491408 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 827/1641] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 1 14:58:22 2022] PUT /identity/v3/projects/2e452af76635415ebc7b06a4b9f625e6/users/8313040cb0214c06979985a9a5f6e5ab/roles/2a2adb769e55476b82e5014897e0a9c0 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:58:22.523993 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-576e0643-4cd8-4b30-a6b0-be49fca5055f None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:58:22.525050 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798b9902-da16-40e8-81c9-29b9f07d3347 None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:22.525342 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798b9902-da16-40e8-81c9-29b9f07d3347 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:22.525513 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-576e0643-4cd8-4b30-a6b0-be49fca5055f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:58:22.525625 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798b9902-da16-40e8-81c9-29b9f07d3347 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:22.617527 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-798b9902-da16-40e8-81c9-29b9f07d3347 None None] MFA Rules not processed for user `8313040cb0214c06979985a9a5f6e5ab`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:58:22.619123 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-798b9902-da16-40e8-81c9-29b9f07d3347 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:58:22.642584 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 815/1642] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 14:58:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:58:22.647009 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570e544a-471e-4085-9fae-c542cc9a7da4 None None] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:22.647628 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570e544a-471e-4085-9fae-c542cc9a7da4 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:22.647811 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570e544a-471e-4085-9fae-c542cc9a7da4 None None] PATH_INFO: `/` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:22.649555 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 816/1643] 10.222.0.33 () {54 vars in 962 bytes} [Thu Sep 1 14:58:22 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:58:22.656496 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aaf89b9-1f71-42e8-a7b8-1c4b70b35a9f None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:22.656781 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aaf89b9-1f71-42e8-a7b8-1c4b70b35a9f None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:22.657054 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aaf89b9-1f71-42e8-a7b8-1c4b70b35a9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:22.723687 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-5aaf89b9-1f71-42e8-a7b8-1c4b70b35a9f None None] MFA Rules not processed for user `4858bb8621854ec8aba836e7d9050498`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:58:22.727304 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-5aaf89b9-1f71-42e8-a7b8-1c4b70b35a9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:58:22.773947 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 828/1644] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 14:58:22 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:58:22.773947 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798b9902-da16-40e8-81c9-29b9f07d3347 None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:58:22.777848 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-798b9902-da16-40e8-81c9-29b9f07d3347 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:58:22.802891 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:58:22.873086 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798b9902-da16-40e8-81c9-29b9f07d3347 None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:58:22.878998 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 817/1645] 10.222.0.33 () {58 vars in 1059 bytes} [Thu Sep 1 14:58:22 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:58:22.881143 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aaf89b9-1f71-42e8-a7b8-1c4b70b35a9f None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:58:22.883799 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-5aaf89b9-1f71-42e8-a7b8-1c4b70b35a9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:58:22.884088 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fd05b5-1af8-4d36-bacc-dea8f5d0573e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2790149639aa418bbc4e4982e93eda44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:58:22.885289 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fd05b5-1af8-4d36-bacc-dea8f5d0573e service neutron] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:22.885750 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fd05b5-1af8-4d36-bacc-dea8f5d0573e service neutron] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:22.886036 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fd05b5-1af8-4d36-bacc-dea8f5d0573e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:22.902485 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fd05b5-1af8-4d36-bacc-dea8f5d0573e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:58:22.903804 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aaf89b9-1f71-42e8-a7b8-1c4b70b35a9f None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:58:22.905155 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fd05b5-1af8-4d36-bacc-dea8f5d0573e service neutron] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:58:22.912217 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3815e2ee-a086-4868-9517-98b2a1262364 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:58:22.913496 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3815e2ee-a086-4868-9517-98b2a1262364 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:22.914038 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3815e2ee-a086-4868-9517-98b2a1262364 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:22.914531 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3815e2ee-a086-4868-9517-98b2a1262364 admin admin] PATH_INFO: `/v3/users/93495b44907140a2a6665ff26d483421` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:22.916269 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3815e2ee-a086-4868-9517-98b2a1262364 admin admin] RBAC: Authorizing `identity:delete_user(user_id=93495b44907140a2a6665ff26d483421)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:58:22.919094 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3815e2ee-a086-4868-9517-98b2a1262364 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:58:22.928746 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 829/1646] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 14:58:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:58:22.930162 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fd05b5-1af8-4d36-bacc-dea8f5d0573e service neutron] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:58:22.931722 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-97fd05b5-1af8-4d36-bacc-dea8f5d0573e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:58:23.003568 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:58:23.025407 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-3815e2ee-a086-4868-9517-98b2a1262364 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '93495b44907140a2a6665ff26d483421'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:58:23.036856 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fd05b5-1af8-4d36-bacc-dea8f5d0573e service neutron] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:58:23.047631 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0155d3-c15b-45a7-a836-6877e3f667c4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3b81739e34e4e648f88b6d73fcb5e46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:58:23.049534 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0155d3-c15b-45a7-a836-6877e3f667c4 service cinder] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:23.052566 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0155d3-c15b-45a7-a836-6877e3f667c4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:23.053701 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0155d3-c15b-45a7-a836-6877e3f667c4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:23.065546 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-3815e2ee-a086-4868-9517-98b2a1262364 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '93495b44907140a2a6665ff26d483421'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:58:23.066688 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-3815e2ee-a086-4868-9517-98b2a1262364 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '93495b44907140a2a6665ff26d483421'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:58:23.068921 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0155d3-c15b-45a7-a836-6877e3f667c4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:58:23.070809 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0155d3-c15b-45a7-a836-6877e3f667c4 service cinder] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:58:23.094118 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 830/1647] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 14:58:22 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:58:23.097564 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a0cffc-55fd-476f-8d23-bba1770b0dc6 None None] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:23.098524 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-3815e2ee-a086-4868-9517-98b2a1262364 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '93495b44907140a2a6665ff26d483421'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:58:23.099180 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a0cffc-55fd-476f-8d23-bba1770b0dc6 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:23.100032 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a0cffc-55fd-476f-8d23-bba1770b0dc6 None None] PATH_INFO: `/` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:23.100435 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-3815e2ee-a086-4868-9517-98b2a1262364 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '93495b44907140a2a6665ff26d483421'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:58:23.104543 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 831/1648] 10.222.0.33 () {54 vars in 912 bytes} [Thu Sep 1 14:58:23 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:58:23.110020 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26acda31-4341-49ea-ad00-abd8b569c8a4 None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:23.110989 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26acda31-4341-49ea-ad00-abd8b569c8a4 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:23.111533 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26acda31-4341-49ea-ad00-abd8b569c8a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:23.133792 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 818/1649] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Sep 1 14:58:22 2022] DELETE /identity/v3/users/93495b44907140a2a6665ff26d483421 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:58:23.139347 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67cb3f6-ebc0-4e3a-a989-1785c8239ee4 None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:23.139680 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67cb3f6-ebc0-4e3a-a989-1785c8239ee4 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:23.139959 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67cb3f6-ebc0-4e3a-a989-1785c8239ee4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:23.162578 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-26acda31-4341-49ea-ad00-abd8b569c8a4 None None] MFA Rules not processed for user `4858bb8621854ec8aba836e7d9050498`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:58:23.165602 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-26acda31-4341-49ea-ad00-abd8b569c8a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:58:23.195308 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-b67cb3f6-ebc0-4e3a-a989-1785c8239ee4 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:58:23.196719 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-b67cb3f6-ebc0-4e3a-a989-1785c8239ee4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:58:23.276900 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 819/1650] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 14:58:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:58:23.292319 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67cb3f6-ebc0-4e3a-a989-1785c8239ee4 None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:58:23.298321 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-b67cb3f6-ebc0-4e3a-a989-1785c8239ee4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:58:23.378770 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 832/1651] 10.222.0.33 () {58 vars in 1009 bytes} [Thu Sep 1 14:58:23 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 270 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:58:23.392046 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26acda31-4341-49ea-ad00-abd8b569c8a4 None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:58:23.394978 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-26acda31-4341-49ea-ad00-abd8b569c8a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:58:23.414377 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:58:23.464352 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67cb3f6-ebc0-4e3a-a989-1785c8239ee4 None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:58:23.473025 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2836d58-8ed7-4af7-8ddc-26e75ae5330c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2790149639aa418bbc4e4982e93eda44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:58:23.473771 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2836d58-8ed7-4af7-8ddc-26e75ae5330c service neutron] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:23.475872 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2836d58-8ed7-4af7-8ddc-26e75ae5330c service neutron] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:23.477657 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2836d58-8ed7-4af7-8ddc-26e75ae5330c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:23.504145 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2836d58-8ed7-4af7-8ddc-26e75ae5330c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:58:23.509279 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2836d58-8ed7-4af7-8ddc-26e75ae5330c service neutron] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:58:23.542477 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 820/1652] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 14:58:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 251 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:58:23.547351 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26acda31-4341-49ea-ad00-abd8b569c8a4 None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:58:23.553082 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcb1dc1-7fbc-4d5a-904f-84262784f805 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3b81739e34e4e648f88b6d73fcb5e46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:58:23.559186 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcb1dc1-7fbc-4d5a-904f-84262784f805 service cinder] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:23.559481 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcb1dc1-7fbc-4d5a-904f-84262784f805 service cinder] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:23.562218 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcb1dc1-7fbc-4d5a-904f-84262784f805 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:23.589749 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcb1dc1-7fbc-4d5a-904f-84262784f805 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:58:23.599671 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcb1dc1-7fbc-4d5a-904f-84262784f805 service cinder] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:58:23.637014 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 833/1653] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 14:58:23 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 245 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:58:23.871740 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac5b3f7-63ce-40c4-859c-c664afc29af5 None None] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:23.872070 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac5b3f7-63ce-40c4-859c-c664afc29af5 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:23.872281 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac5b3f7-63ce-40c4-859c-c664afc29af5 None None] PATH_INFO: `/` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:23.874280 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 821/1654] 10.222.0.33 () {54 vars in 912 bytes} [Thu Sep 1 14:58:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:58:23.878840 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcb1dc1-7fbc-4d5a-904f-84262784f805 service cinder] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:58:23.880405 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-1fcb1dc1-7fbc-4d5a-904f-84262784f805 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:58:23.901397 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ff4e13-0389-4980-a000-8f0b7dc20eec None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:23.901683 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ff4e13-0389-4980-a000-8f0b7dc20eec None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:23.901973 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ff4e13-0389-4980-a000-8f0b7dc20eec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:23.904052 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcb1dc1-7fbc-4d5a-904f-84262784f805 service cinder] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:58:23.912583 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61aaff89-9cfa-45a6-b409-8093bac00776 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:58:23.913238 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61aaff89-9cfa-45a6-b409-8093bac00776 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:23.913697 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61aaff89-9cfa-45a6-b409-8093bac00776 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:23.913974 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61aaff89-9cfa-45a6-b409-8093bac00776 admin admin] PATH_INFO: `/v3/users/76b06e441b9f45f99a80bce50f890f00` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:23.915865 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61aaff89-9cfa-45a6-b409-8093bac00776 admin admin] RBAC: Authorizing `identity:delete_user(user_id=76b06e441b9f45f99a80bce50f890f00)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:58:23.916905 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: WARNING py.warnings [None req-61aaff89-9cfa-45a6-b409-8093bac00776 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 01 14:58:23.916905 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: warnings.warn(msg) Sep 01 14:58:23.916905 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: Sep 01 14:58:23.918458 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61aaff89-9cfa-45a6-b409-8093bac00776 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:58:23.944451 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-b5ff4e13-0389-4980-a000-8f0b7dc20eec None None] MFA Rules not processed for user `4858bb8621854ec8aba836e7d9050498`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:58:23.947033 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-b5ff4e13-0389-4980-a000-8f0b7dc20eec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:58:23.976482 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 822/1655] 10.222.0.33 () {58 vars in 1009 bytes} [Thu Sep 1 14:58:23 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:58:23.992407 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ff4e13-0389-4980-a000-8f0b7dc20eec None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:58:24.004543 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ff4e13-0389-4980-a000-8f0b7dc20eec None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:58:24.013915 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba886738-b1e4-489a-8cf7-05818886bc47 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3b81739e34e4e648f88b6d73fcb5e46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:58:24.014574 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba886738-b1e4-489a-8cf7-05818886bc47 service cinder] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:24.014847 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba886738-b1e4-489a-8cf7-05818886bc47 service cinder] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:24.015116 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba886738-b1e4-489a-8cf7-05818886bc47 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:24.015771 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-61aaff89-9cfa-45a6-b409-8093bac00776 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '76b06e441b9f45f99a80bce50f890f00'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:58:24.036015 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-61aaff89-9cfa-45a6-b409-8093bac00776 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '76b06e441b9f45f99a80bce50f890f00'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:58:24.037892 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-61aaff89-9cfa-45a6-b409-8093bac00776 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '76b06e441b9f45f99a80bce50f890f00'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:58:24.051288 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba886738-b1e4-489a-8cf7-05818886bc47 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:58:24.053172 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba886738-b1e4-489a-8cf7-05818886bc47 service cinder] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:58:24.069027 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:58:24.077519 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 823/1656] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 14:58:23 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 14:58:24.085963 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-61aaff89-9cfa-45a6-b409-8093bac00776 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '76b06e441b9f45f99a80bce50f890f00'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:58:24.090816 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-61aaff89-9cfa-45a6-b409-8093bac00776 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '76b06e441b9f45f99a80bce50f890f00'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:58:24.137066 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 834/1657] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Sep 1 14:58:23 2022] DELETE /identity/v3/users/76b06e441b9f45f99a80bce50f890f00 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:58:24.142121 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43feffec-e89a-4607-8758-a799ffa9ad3e None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:24.142715 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43feffec-e89a-4607-8758-a799ffa9ad3e None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:24.142999 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43feffec-e89a-4607-8758-a799ffa9ad3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:24.172789 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-43feffec-e89a-4607-8758-a799ffa9ad3e None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:58:24.174911 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-43feffec-e89a-4607-8758-a799ffa9ad3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:58:24.287833 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61aaff89-9cfa-45a6-b409-8093bac00776 admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:58:24.291569 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-61aaff89-9cfa-45a6-b409-8093bac00776 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:58:24.344278 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 824/1658] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 14:58:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:58:24.402733 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61aaff89-9cfa-45a6-b409-8093bac00776 admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:58:24.408979 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897504aa-39be-4a04-84cc-8a43dd884851 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:58:24.409964 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897504aa-39be-4a04-84cc-8a43dd884851 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:24.410402 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897504aa-39be-4a04-84cc-8a43dd884851 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:24.410676 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897504aa-39be-4a04-84cc-8a43dd884851 admin admin] PATH_INFO: `/v3/users/c57d26d885514890b8b9d73548e81325` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:24.412631 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897504aa-39be-4a04-84cc-8a43dd884851 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c57d26d885514890b8b9d73548e81325)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:58:24.415675 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897504aa-39be-4a04-84cc-8a43dd884851 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:58:24.452101 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43feffec-e89a-4607-8758-a799ffa9ad3e None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:58:24.453903 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-43feffec-e89a-4607-8758-a799ffa9ad3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:58:24.527243 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-897504aa-39be-4a04-84cc-8a43dd884851 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c57d26d885514890b8b9d73548e81325'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:58:24.544215 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-897504aa-39be-4a04-84cc-8a43dd884851 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c57d26d885514890b8b9d73548e81325'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:58:24.544877 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-897504aa-39be-4a04-84cc-8a43dd884851 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c57d26d885514890b8b9d73548e81325'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:58:24.583889 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-897504aa-39be-4a04-84cc-8a43dd884851 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c57d26d885514890b8b9d73548e81325'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:58:24.584822 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-897504aa-39be-4a04-84cc-8a43dd884851 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c57d26d885514890b8b9d73548e81325'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:58:24.625000 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 835/1659] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Sep 1 14:58:24 2022] DELETE /identity/v3/users/c57d26d885514890b8b9d73548e81325 => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:58:24.631011 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f1a29e-04b5-4788-9388-2b65d0c7c386 None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:24.631516 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f1a29e-04b5-4788-9388-2b65d0c7c386 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:24.631941 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f1a29e-04b5-4788-9388-2b65d0c7c386 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:24.636581 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43feffec-e89a-4607-8758-a799ffa9ad3e None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:58:24.684047 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-f5f1a29e-04b5-4788-9388-2b65d0c7c386 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:58:24.687310 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-f5f1a29e-04b5-4788-9388-2b65d0c7c386 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:58:24.701440 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c58f001-120a-425f-8400-0864bd35d06c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2790149639aa418bbc4e4982e93eda44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:58:24.702402 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c58f001-120a-425f-8400-0864bd35d06c service neutron] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:24.702855 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c58f001-120a-425f-8400-0864bd35d06c service neutron] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:24.703332 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c58f001-120a-425f-8400-0864bd35d06c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:24.705374 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-7c58f001-120a-425f-8400-0864bd35d06c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:58:24.742020 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:58:24.783130 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c58f001-120a-425f-8400-0864bd35d06c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:58:24.823220 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c58f001-120a-425f-8400-0864bd35d06c service neutron] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:58:24.823220 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 825/1660] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 14:58:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:58:24.823220 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c58f001-120a-425f-8400-0864bd35d06c service neutron] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:58:24.823220 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-7c58f001-120a-425f-8400-0864bd35d06c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:58:24.835466 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c58f001-120a-425f-8400-0864bd35d06c service neutron] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:58:24.841739 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f3a0f2-6a3e-4af2-b360-06095bc3c799 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:58:24.842632 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f3a0f2-6a3e-4af2-b360-06095bc3c799 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:24.843158 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f3a0f2-6a3e-4af2-b360-06095bc3c799 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:24.843586 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f3a0f2-6a3e-4af2-b360-06095bc3c799 admin admin] PATH_INFO: `/v3/projects/addc36256453406597172eeacd3d911e` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:24.845700 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f3a0f2-6a3e-4af2-b360-06095bc3c799 admin admin] RBAC: Authorizing `identity:delete_project(project_id=addc36256453406597172eeacd3d911e)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:58:24.846951 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: WARNING py.warnings [None req-01f3a0f2-6a3e-4af2-b360-06095bc3c799 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 01 14:58:24.846951 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: warnings.warn(msg) Sep 01 14:58:24.846951 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: Sep 01 14:58:24.849531 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f3a0f2-6a3e-4af2-b360-06095bc3c799 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:58:24.890311 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 836/1661] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 14:58:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:58:24.927594 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-01f3a0f2-6a3e-4af2-b360-06095bc3c799 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'addc36256453406597172eeacd3d911e', 'request_id': 'req-01f3a0f2-6a3e-4af2-b360-06095bc3c799'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:58:24.930771 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f1a29e-04b5-4788-9388-2b65d0c7c386 None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:58:24.933588 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-f5f1a29e-04b5-4788-9388-2b65d0c7c386 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:58:24.944496 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:58:24.954861 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f1a29e-04b5-4788-9388-2b65d0c7c386 None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:58:24.968536 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-01f3a0f2-6a3e-4af2-b360-06095bc3c799 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'addc36256453406597172eeacd3d911e', 'request_id': 'req-01f3a0f2-6a3e-4af2-b360-06095bc3c799'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:58:24.981497 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-01f3a0f2-6a3e-4af2-b360-06095bc3c799 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'addc36256453406597172eeacd3d911e', 'request_id': 'req-01f3a0f2-6a3e-4af2-b360-06095bc3c799'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:58:25.007740 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-01f3a0f2-6a3e-4af2-b360-06095bc3c799 admin admin] The token cache is being invalidate because project addc36256453406597172eeacd3d911e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81341) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:58:25.009738 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-01f3a0f2-6a3e-4af2-b360-06095bc3c799 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:58:25.011306 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-01f3a0f2-6a3e-4af2-b360-06095bc3c799 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:58:25.016669 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 826/1662] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Sep 1 14:58:24 2022] DELETE /identity/v3/projects/addc36256453406597172eeacd3d911e => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:58:25.021502 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f3a0f2-6a3e-4af2-b360-06095bc3c799 admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:58:25.024294 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-01f3a0f2-6a3e-4af2-b360-06095bc3c799 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:58:25.089964 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9005b7-6a3c-4351-9727-40ac848f7948 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2790149639aa418bbc4e4982e93eda44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:58:25.090870 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9005b7-6a3c-4351-9727-40ac848f7948 service neutron] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:25.092950 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9005b7-6a3c-4351-9727-40ac848f7948 service neutron] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:25.093338 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9005b7-6a3c-4351-9727-40ac848f7948 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:25.100117 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-0a9005b7-6a3c-4351-9727-40ac848f7948 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:58:25.192163 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f3a0f2-6a3e-4af2-b360-06095bc3c799 admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:58:25.203186 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b51cb1-cffe-4d17-8b73-380bbbe50775 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:58:25.206217 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b51cb1-cffe-4d17-8b73-380bbbe50775 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:25.206969 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b51cb1-cffe-4d17-8b73-380bbbe50775 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:25.207672 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b51cb1-cffe-4d17-8b73-380bbbe50775 admin admin] PATH_INFO: `/v3/users/8e4b3ada9f154e839231d5ae2904a42b` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:25.211296 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b51cb1-cffe-4d17-8b73-380bbbe50775 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8e4b3ada9f154e839231d5ae2904a42b)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:58:25.214927 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b51cb1-cffe-4d17-8b73-380bbbe50775 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:58:25.269648 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9005b7-6a3c-4351-9727-40ac848f7948 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:58:25.272544 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9005b7-6a3c-4351-9727-40ac848f7948 service neutron] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:58:25.294542 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 837/1663] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 14:58:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 14:58:25.298219 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9005b7-6a3c-4351-9727-40ac848f7948 service neutron] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:58:25.299872 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-0a9005b7-6a3c-4351-9727-40ac848f7948 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:58:25.313362 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9005b7-6a3c-4351-9727-40ac848f7948 service neutron] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:58:25.318804 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94f8d14-f57b-4764-985e-37edf4ae565b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:58:25.319708 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94f8d14-f57b-4764-985e-37edf4ae565b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:25.320125 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94f8d14-f57b-4764-985e-37edf4ae565b admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:25.320569 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94f8d14-f57b-4764-985e-37edf4ae565b admin admin] PATH_INFO: `/v3/projects/1d77f3fbed06493b8c31048594ec3020` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:25.330436 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-f1b51cb1-cffe-4d17-8b73-380bbbe50775 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8e4b3ada9f154e839231d5ae2904a42b'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:58:25.332064 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94f8d14-f57b-4764-985e-37edf4ae565b admin admin] RBAC: Authorizing `identity:delete_project(project_id=1d77f3fbed06493b8c31048594ec3020)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:58:25.334872 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94f8d14-f57b-4764-985e-37edf4ae565b admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:58:25.335603 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:58:25.350277 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-f1b51cb1-cffe-4d17-8b73-380bbbe50775 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8e4b3ada9f154e839231d5ae2904a42b'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:58:25.351022 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-f1b51cb1-cffe-4d17-8b73-380bbbe50775 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8e4b3ada9f154e839231d5ae2904a42b'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:58:25.375409 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-f1b51cb1-cffe-4d17-8b73-380bbbe50775 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8e4b3ada9f154e839231d5ae2904a42b'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:58:25.376340 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-f1b51cb1-cffe-4d17-8b73-380bbbe50775 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8e4b3ada9f154e839231d5ae2904a42b'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:58:25.404920 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-b94f8d14-f57b-4764-985e-37edf4ae565b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1d77f3fbed06493b8c31048594ec3020', 'request_id': 'req-b94f8d14-f57b-4764-985e-37edf4ae565b'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:58:25.407119 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 827/1664] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Sep 1 14:58:25 2022] DELETE /identity/v3/users/8e4b3ada9f154e839231d5ae2904a42b => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:58:25.418178 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-b94f8d14-f57b-4764-985e-37edf4ae565b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1d77f3fbed06493b8c31048594ec3020', 'request_id': 'req-b94f8d14-f57b-4764-985e-37edf4ae565b'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:58:25.431553 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-b94f8d14-f57b-4764-985e-37edf4ae565b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1d77f3fbed06493b8c31048594ec3020', 'request_id': 'req-b94f8d14-f57b-4764-985e-37edf4ae565b'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:58:25.447794 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-b94f8d14-f57b-4764-985e-37edf4ae565b admin admin] The token cache is being invalidate because project 1d77f3fbed06493b8c31048594ec3020 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81342) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:58:25.448293 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-b94f8d14-f57b-4764-985e-37edf4ae565b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:58:25.449193 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-b94f8d14-f57b-4764-985e-37edf4ae565b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:58:25.452746 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 838/1665] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Sep 1 14:58:25 2022] DELETE /identity/v3/projects/1d77f3fbed06493b8c31048594ec3020 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:58:25.457126 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b51cb1-cffe-4d17-8b73-380bbbe50775 admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:58:25.459938 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-f1b51cb1-cffe-4d17-8b73-380bbbe50775 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:58:25.585374 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b51cb1-cffe-4d17-8b73-380bbbe50775 admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:58:25.591044 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c40b8d9-2673-43a4-8a31-843568ee420b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:58:25.592125 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c40b8d9-2673-43a4-8a31-843568ee420b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:25.592509 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c40b8d9-2673-43a4-8a31-843568ee420b admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:25.593148 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c40b8d9-2673-43a4-8a31-843568ee420b admin admin] PATH_INFO: `/v3/users/df61e99ad24c4d398ad96980af6c2b57` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:25.595196 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c40b8d9-2673-43a4-8a31-843568ee420b admin admin] RBAC: Authorizing `identity:delete_user(user_id=df61e99ad24c4d398ad96980af6c2b57)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:58:25.597280 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c40b8d9-2673-43a4-8a31-843568ee420b admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:58:25.682089 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-1c40b8d9-2673-43a4-8a31-843568ee420b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'df61e99ad24c4d398ad96980af6c2b57'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:58:25.703456 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-1c40b8d9-2673-43a4-8a31-843568ee420b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'df61e99ad24c4d398ad96980af6c2b57'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:58:25.704249 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-1c40b8d9-2673-43a4-8a31-843568ee420b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'df61e99ad24c4d398ad96980af6c2b57'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:58:25.724740 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-1c40b8d9-2673-43a4-8a31-843568ee420b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'df61e99ad24c4d398ad96980af6c2b57'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:58:25.725561 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-1c40b8d9-2673-43a4-8a31-843568ee420b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'df61e99ad24c4d398ad96980af6c2b57'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:58:25.746402 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 828/1666] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Sep 1 14:58:25 2022] DELETE /identity/v3/users/df61e99ad24c4d398ad96980af6c2b57 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:58:26.676446 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94f8d14-f57b-4764-985e-37edf4ae565b admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:58:26.678001 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-b94f8d14-f57b-4764-985e-37edf4ae565b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:58:26.686330 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c40b8d9-2673-43a4-8a31-843568ee420b admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:58:26.687835 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-1c40b8d9-2673-43a4-8a31-843568ee420b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:58:26.808220 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:58:26.852422 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c40b8d9-2673-43a4-8a31-843568ee420b admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:58:26.853168 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:58:26.866410 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e23f99-9645-4076-a780-ecf1d237ffdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:58:26.873183 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e23f99-9645-4076-a780-ecf1d237ffdc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:26.873598 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e23f99-9645-4076-a780-ecf1d237ffdc admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:26.875292 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e23f99-9645-4076-a780-ecf1d237ffdc admin admin] PATH_INFO: `/v3/projects/9af049e9440a458e96c0bdab2c71b5e0` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:26.888804 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e23f99-9645-4076-a780-ecf1d237ffdc admin admin] RBAC: Authorizing `identity:delete_project(project_id=9af049e9440a458e96c0bdab2c71b5e0)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:58:26.896963 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e23f99-9645-4076-a780-ecf1d237ffdc admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:58:26.923151 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94f8d14-f57b-4764-985e-37edf4ae565b admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:58:26.939268 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83001690-bc19-4d13-b627-a53c3245b62e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:58:26.939932 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83001690-bc19-4d13-b627-a53c3245b62e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:26.940214 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83001690-bc19-4d13-b627-a53c3245b62e admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:26.940489 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83001690-bc19-4d13-b627-a53c3245b62e admin admin] PATH_INFO: `/v3/projects/ae1a3cdeba4546b2b30ecdaf48ccd8cb` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:26.941693 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83001690-bc19-4d13-b627-a53c3245b62e admin admin] RBAC: Authorizing `identity:delete_project(project_id=ae1a3cdeba4546b2b30ecdaf48ccd8cb)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:58:26.943725 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83001690-bc19-4d13-b627-a53c3245b62e admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:58:27.029567 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-b7e23f99-9645-4076-a780-ecf1d237ffdc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9af049e9440a458e96c0bdab2c71b5e0', 'request_id': 'req-b7e23f99-9645-4076-a780-ecf1d237ffdc'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:58:27.056352 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-b7e23f99-9645-4076-a780-ecf1d237ffdc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9af049e9440a458e96c0bdab2c71b5e0', 'request_id': 'req-b7e23f99-9645-4076-a780-ecf1d237ffdc'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:58:27.079442 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-b7e23f99-9645-4076-a780-ecf1d237ffdc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9af049e9440a458e96c0bdab2c71b5e0', 'request_id': 'req-b7e23f99-9645-4076-a780-ecf1d237ffdc'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:58:27.080589 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-83001690-bc19-4d13-b627-a53c3245b62e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ae1a3cdeba4546b2b30ecdaf48ccd8cb', 'request_id': 'req-83001690-bc19-4d13-b627-a53c3245b62e'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:58:27.101299 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-b7e23f99-9645-4076-a780-ecf1d237ffdc admin admin] The token cache is being invalidate because project 9af049e9440a458e96c0bdab2c71b5e0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81341) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:58:27.101646 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-b7e23f99-9645-4076-a780-ecf1d237ffdc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:58:27.103066 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-b7e23f99-9645-4076-a780-ecf1d237ffdc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:58:27.107186 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-83001690-bc19-4d13-b627-a53c3245b62e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ae1a3cdeba4546b2b30ecdaf48ccd8cb', 'request_id': 'req-83001690-bc19-4d13-b627-a53c3245b62e'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:58:27.108398 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 829/1667] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Sep 1 14:58:26 2022] DELETE /identity/v3/projects/9af049e9440a458e96c0bdab2c71b5e0 => generated 0 bytes in 422 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:58:27.113992 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e23f99-9645-4076-a780-ecf1d237ffdc admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:58:27.117101 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-b7e23f99-9645-4076-a780-ecf1d237ffdc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:58:27.131025 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-83001690-bc19-4d13-b627-a53c3245b62e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ae1a3cdeba4546b2b30ecdaf48ccd8cb', 'request_id': 'req-83001690-bc19-4d13-b627-a53c3245b62e'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:58:27.148463 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-83001690-bc19-4d13-b627-a53c3245b62e admin admin] The token cache is being invalidate because project ae1a3cdeba4546b2b30ecdaf48ccd8cb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81342) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:58:27.149090 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-83001690-bc19-4d13-b627-a53c3245b62e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:58:27.150560 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-83001690-bc19-4d13-b627-a53c3245b62e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:58:27.154307 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 839/1668] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Sep 1 14:58:26 2022] DELETE /identity/v3/projects/ae1a3cdeba4546b2b30ecdaf48ccd8cb => generated 0 bytes in 478 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:58:27.155485 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83001690-bc19-4d13-b627-a53c3245b62e admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:58:27.167251 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-83001690-bc19-4d13-b627-a53c3245b62e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:58:27.309230 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e23f99-9645-4076-a780-ecf1d237ffdc admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:58:27.361308 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83001690-bc19-4d13-b627-a53c3245b62e admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:58:27.376099 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c7bcc5-b132-427b-8e93-f2641a717dae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:58:27.376754 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c7bcc5-b132-427b-8e93-f2641a717dae admin admin] REQUEST_METHOD: `DELETE` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:27.377033 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c7bcc5-b132-427b-8e93-f2641a717dae admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:27.377309 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c7bcc5-b132-427b-8e93-f2641a717dae admin admin] PATH_INFO: `/v3/projects/54e5a154e23440ce95f4644b4e2ee73f` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:27.383329 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c7bcc5-b132-427b-8e93-f2641a717dae admin admin] RBAC: Authorizing `identity:delete_project(project_id=54e5a154e23440ce95f4644b4e2ee73f)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:58:27.385310 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c7bcc5-b132-427b-8e93-f2641a717dae admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:58:27.432235 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f76d6ee-a36d-44fe-9228-c258e062d53b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:58:27.432975 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f76d6ee-a36d-44fe-9228-c258e062d53b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:27.433920 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f76d6ee-a36d-44fe-9228-c258e062d53b admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:27.434208 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f76d6ee-a36d-44fe-9228-c258e062d53b admin admin] PATH_INFO: `/v3/users/8313040cb0214c06979985a9a5f6e5ab` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:27.440112 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f76d6ee-a36d-44fe-9228-c258e062d53b admin admin] RBAC: Authorizing `identity:delete_user(user_id=8313040cb0214c06979985a9a5f6e5ab)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:58:27.443182 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f76d6ee-a36d-44fe-9228-c258e062d53b admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:58:27.495287 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-04c7bcc5-b132-427b-8e93-f2641a717dae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '54e5a154e23440ce95f4644b4e2ee73f', 'request_id': 'req-04c7bcc5-b132-427b-8e93-f2641a717dae'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:58:27.509537 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-04c7bcc5-b132-427b-8e93-f2641a717dae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '54e5a154e23440ce95f4644b4e2ee73f', 'request_id': 'req-04c7bcc5-b132-427b-8e93-f2641a717dae'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:58:27.522483 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:58:27.530345 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-04c7bcc5-b132-427b-8e93-f2641a717dae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '54e5a154e23440ce95f4644b4e2ee73f', 'request_id': 'req-04c7bcc5-b132-427b-8e93-f2641a717dae'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:58:27.539265 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:58:27.556152 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-04c7bcc5-b132-427b-8e93-f2641a717dae admin admin] The token cache is being invalidate because project 54e5a154e23440ce95f4644b4e2ee73f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81342) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:58:27.556619 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-04c7bcc5-b132-427b-8e93-f2641a717dae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:58:27.559965 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-04c7bcc5-b132-427b-8e93-f2641a717dae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:58:27.562704 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 840/1669] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Sep 1 14:58:27 2022] DELETE /identity/v3/projects/54e5a154e23440ce95f4644b4e2ee73f => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:58:27.569879 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65d59e0-efd6-44a3-995a-8690aa86efeb None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:27.570736 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65d59e0-efd6-44a3-995a-8690aa86efeb None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:27.571820 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65d59e0-efd6-44a3-995a-8690aa86efeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:27.572457 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-1f76d6ee-a36d-44fe-9228-c258e062d53b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8313040cb0214c06979985a9a5f6e5ab'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:58:27.586598 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-1f76d6ee-a36d-44fe-9228-c258e062d53b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8313040cb0214c06979985a9a5f6e5ab'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:58:27.587440 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-1f76d6ee-a36d-44fe-9228-c258e062d53b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8313040cb0214c06979985a9a5f6e5ab'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:58:27.607174 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-1f76d6ee-a36d-44fe-9228-c258e062d53b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8313040cb0214c06979985a9a5f6e5ab'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:58:27.608229 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-1f76d6ee-a36d-44fe-9228-c258e062d53b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8313040cb0214c06979985a9a5f6e5ab'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:58:27.638096 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-b65d59e0-efd6-44a3-995a-8690aa86efeb None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:58:27.643986 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 830/1670] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Sep 1 14:58:27 2022] DELETE /identity/v3/users/8313040cb0214c06979985a9a5f6e5ab => generated 0 bytes in 528 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:58:27.643986 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e750b3f-6207-4f46-afc5-3746e7ffa0a7 None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:27.643986 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e750b3f-6207-4f46-afc5-3746e7ffa0a7 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:27.643986 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e750b3f-6207-4f46-afc5-3746e7ffa0a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:27.644493 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-b65d59e0-efd6-44a3-995a-8690aa86efeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:58:27.682203 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-5e750b3f-6207-4f46-afc5-3746e7ffa0a7 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:58:27.683625 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-5e750b3f-6207-4f46-afc5-3746e7ffa0a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:58:27.884562 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 831/1671] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 14:58:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:58:27.886376 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e750b3f-6207-4f46-afc5-3746e7ffa0a7 None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:58:27.894520 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-5e750b3f-6207-4f46-afc5-3746e7ffa0a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:58:27.922324 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 841/1672] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 14:58:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 359 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:58:27.923888 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6674f35e-5172-4d27-98b3-08129df8815b None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:27.924180 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6674f35e-5172-4d27-98b3-08129df8815b None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:27.924454 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6674f35e-5172-4d27-98b3-08129df8815b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:27.949997 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-6674f35e-5172-4d27-98b3-08129df8815b None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:58:27.951340 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-6674f35e-5172-4d27-98b3-08129df8815b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:58:27.952366 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e750b3f-6207-4f46-afc5-3746e7ffa0a7 None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:58:27.961546 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 842/1673] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 14:58:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:58:27.964767 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c7ca42-d734-4eb5-b3c6-25dcfe81c441 None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:27.965158 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c7ca42-d734-4eb5-b3c6-25dcfe81c441 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:27.965435 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c7ca42-d734-4eb5-b3c6-25dcfe81c441 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:27.987377 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e362d2d-4a39-4e0e-8f21-4ce5a957fd54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:58:27.987803 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e362d2d-4a39-4e0e-8f21-4ce5a957fd54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:27.988336 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e362d2d-4a39-4e0e-8f21-4ce5a957fd54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:27.988737 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e362d2d-4a39-4e0e-8f21-4ce5a957fd54 admin admin] PATH_INFO: `/v3/projects/4a4b43c191584f83a131556d29c0b0f7` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:27.995120 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e362d2d-4a39-4e0e-8f21-4ce5a957fd54 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4a4b43c191584f83a131556d29c0b0f7)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:58:27.997439 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e362d2d-4a39-4e0e-8f21-4ce5a957fd54 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:58:28.011097 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-c5c7ca42-d734-4eb5-b3c6-25dcfe81c441 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:58:28.012351 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-c5c7ca42-d734-4eb5-b3c6-25dcfe81c441 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:58:28.025554 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 843/1674] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 14:58:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:58:28.026638 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c7ca42-d734-4eb5-b3c6-25dcfe81c441 None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:58:28.040189 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c7ca42-d734-4eb5-b3c6-25dcfe81c441 None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:58:28.045041 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d220e21a-9a83-4616-b265-d7b3f1615ed9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:58:28.045960 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d220e21a-9a83-4616-b265-d7b3f1615ed9 admin admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:28.046265 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d220e21a-9a83-4616-b265-d7b3f1615ed9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:28.046542 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d220e21a-9a83-4616-b265-d7b3f1615ed9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:28.047313 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d220e21a-9a83-4616-b265-d7b3f1615ed9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:58:28.047620 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d220e21a-9a83-4616-b265-d7b3f1615ed9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:58:28.049849 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d220e21a-9a83-4616-b265-d7b3f1615ed9 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:58:28.081092 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 844/1675] 10.222.0.33 () {58 vars in 1153 bytes} [Thu Sep 1 14:58:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:58:28.082144 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d220e21a-9a83-4616-b265-d7b3f1615ed9 admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:58:28.106916 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d220e21a-9a83-4616-b265-d7b3f1615ed9 admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:58:28.120694 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5371cf89-ebc9-434e-8ca6-3cf17e47cc2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:58:28.125423 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5371cf89-ebc9-434e-8ca6-3cf17e47cc2e admin admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:28.126576 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5371cf89-ebc9-434e-8ca6-3cf17e47cc2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:28.126862 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5371cf89-ebc9-434e-8ca6-3cf17e47cc2e admin admin] PATH_INFO: `/v3/domains` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:28.127558 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5371cf89-ebc9-434e-8ca6-3cf17e47cc2e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:58:28.127864 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5371cf89-ebc9-434e-8ca6-3cf17e47cc2e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:58:28.134615 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5371cf89-ebc9-434e-8ca6-3cf17e47cc2e admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:58:28.172673 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 845/1676] 10.222.0.33 () {58 vars in 1153 bytes} [Thu Sep 1 14:58:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:58:28.175452 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5371cf89-ebc9-434e-8ca6-3cf17e47cc2e admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:58:28.175586 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-2e362d2d-4a39-4e0e-8f21-4ce5a957fd54 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4a4b43c191584f83a131556d29c0b0f7', 'request_id': 'req-2e362d2d-4a39-4e0e-8f21-4ce5a957fd54'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:58:28.193391 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-2e362d2d-4a39-4e0e-8f21-4ce5a957fd54 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4a4b43c191584f83a131556d29c0b0f7', 'request_id': 'req-2e362d2d-4a39-4e0e-8f21-4ce5a957fd54'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:58:28.205874 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5371cf89-ebc9-434e-8ca6-3cf17e47cc2e admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:58:28.211628 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-2e362d2d-4a39-4e0e-8f21-4ce5a957fd54 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4a4b43c191584f83a131556d29c0b0f7', 'request_id': 'req-2e362d2d-4a39-4e0e-8f21-4ce5a957fd54'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:58:28.240966 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-2e362d2d-4a39-4e0e-8f21-4ce5a957fd54 admin admin] The token cache is being invalidate because project 4a4b43c191584f83a131556d29c0b0f7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81341) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:58:28.241925 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-2e362d2d-4a39-4e0e-8f21-4ce5a957fd54 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:58:28.244945 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-2e362d2d-4a39-4e0e-8f21-4ce5a957fd54 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:58:28.249186 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 832/1677] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Sep 1 14:58:27 2022] DELETE /identity/v3/projects/4a4b43c191584f83a131556d29c0b0f7 => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:58:28.250152 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e362d2d-4a39-4e0e-8f21-4ce5a957fd54 admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:58:28.252954 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-2e362d2d-4a39-4e0e-8f21-4ce5a957fd54 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:58:28.339302 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:58:28.383441 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be4cff3-f116-4b4d-88fa-2bb6d81e1fee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:58:28.385356 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be4cff3-f116-4b4d-88fa-2bb6d81e1fee admin admin] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:28.385647 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be4cff3-f116-4b4d-88fa-2bb6d81e1fee admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:28.386839 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be4cff3-f116-4b4d-88fa-2bb6d81e1fee admin admin] PATH_INFO: `/v3/projects` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:28.387542 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be4cff3-f116-4b4d-88fa-2bb6d81e1fee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:58:28.393267 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be4cff3-f116-4b4d-88fa-2bb6d81e1fee admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:58:28.443397 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e362d2d-4a39-4e0e-8f21-4ce5a957fd54 admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:58:28.455104 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6d8232-c70a-43d7-a929-29ec6ddbff86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:58:28.455928 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6d8232-c70a-43d7-a929-29ec6ddbff86 admin admin] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:28.456212 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6d8232-c70a-43d7-a929-29ec6ddbff86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:28.456624 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6d8232-c70a-43d7-a929-29ec6ddbff86 admin admin] PATH_INFO: `/v3/projects` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:28.457567 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6d8232-c70a-43d7-a929-29ec6ddbff86 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:58:28.460634 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6d8232-c70a-43d7-a929-29ec6ddbff86 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:58:28.474372 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 846/1678] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Sep 1 14:58:28 2022] POST /identity/v3/projects => generated 373 bytes in 300 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:58:28.475879 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7161d5-36c8-4fcf-88ad-d5e72f2f93a0 None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:28.476169 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7161d5-36c8-4fcf-88ad-d5e72f2f93a0 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:28.476447 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7161d5-36c8-4fcf-88ad-d5e72f2f93a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:28.537616 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-5e7161d5-36c8-4fcf-88ad-d5e72f2f93a0 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:58:28.541317 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-5e7161d5-36c8-4fcf-88ad-d5e72f2f93a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:58:28.544049 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 833/1679] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Sep 1 14:58:28 2022] POST /identity/v3/projects => generated 377 bytes in 294 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:58:28.546257 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6d8232-c70a-43d7-a929-29ec6ddbff86 admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:58:28.549484 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-8c6d8232-c70a-43d7-a929-29ec6ddbff86 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:58:28.620740 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:58:28.650765 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 847/1680] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 14:58:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:58:28.651603 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7161d5-36c8-4fcf-88ad-d5e72f2f93a0 None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:58:28.663409 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7161d5-36c8-4fcf-88ad-d5e72f2f93a0 None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:58:28.669673 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2555818a-519a-4d99-8b1c-fd20f4dbccba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:58:28.670361 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2555818a-519a-4d99-8b1c-fd20f4dbccba admin admin] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:28.671394 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2555818a-519a-4d99-8b1c-fd20f4dbccba admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:28.671679 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2555818a-519a-4d99-8b1c-fd20f4dbccba admin admin] PATH_INFO: `/v3/users` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:28.672480 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2555818a-519a-4d99-8b1c-fd20f4dbccba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:58:28.675116 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2555818a-519a-4d99-8b1c-fd20f4dbccba admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:58:28.709738 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 848/1681] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Sep 1 14:58:28 2022] POST /identity/v3/users => generated 289 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:58:28.711275 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f130608c-c063-4c2d-b1c4-ffbff684ccaf None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:28.711506 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f130608c-c063-4c2d-b1c4-ffbff684ccaf None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:28.711840 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f130608c-c063-4c2d-b1c4-ffbff684ccaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:28.737709 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-f130608c-c063-4c2d-b1c4-ffbff684ccaf None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:58:28.739518 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-f130608c-c063-4c2d-b1c4-ffbff684ccaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:58:28.748017 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6d8232-c70a-43d7-a929-29ec6ddbff86 admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:58:28.753557 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 849/1682] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 14:58:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:58:28.754691 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f130608c-c063-4c2d-b1c4-ffbff684ccaf None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:58:28.758051 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9e4b99-9fe6-4314-88e4-cb6a1a6f7097 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:58:28.759013 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9e4b99-9fe6-4314-88e4-cb6a1a6f7097 admin admin] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:28.759520 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9e4b99-9fe6-4314-88e4-cb6a1a6f7097 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:28.759990 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9e4b99-9fe6-4314-88e4-cb6a1a6f7097 admin admin] PATH_INFO: `/v3/users` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:28.761290 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9e4b99-9fe6-4314-88e4-cb6a1a6f7097 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:58:28.763853 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9e4b99-9fe6-4314-88e4-cb6a1a6f7097 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:58:28.767144 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f130608c-c063-4c2d-b1c4-ffbff684ccaf None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:58:28.774869 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b31964c-b85d-4f67-aa4e-02ebd420ab96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:58:28.775580 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b31964c-b85d-4f67-aa4e-02ebd420ab96 admin admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:28.775914 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b31964c-b85d-4f67-aa4e-02ebd420ab96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:28.776234 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b31964c-b85d-4f67-aa4e-02ebd420ab96 admin admin] PATH_INFO: `/v3/roles` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:28.777371 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b31964c-b85d-4f67-aa4e-02ebd420ab96 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:58:28.779337 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b31964c-b85d-4f67-aa4e-02ebd420ab96 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:58:28.792498 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 850/1683] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 1 14:58:28 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:58:28.793531 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b31964c-b85d-4f67-aa4e-02ebd420ab96 admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:58:28.822032 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b31964c-b85d-4f67-aa4e-02ebd420ab96 admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:58:28.823009 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 834/1684] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Sep 1 14:58:28 2022] POST /identity/v3/users => generated 287 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:58:28.824120 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9e4b99-9fe6-4314-88e4-cb6a1a6f7097 admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:58:28.835257 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2c3280-9f4a-4a33-a496-d06f0ba6c53d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:58:28.836634 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2c3280-9f4a-4a33-a496-d06f0ba6c53d admin admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:28.836918 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2c3280-9f4a-4a33-a496-d06f0ba6c53d admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:28.837190 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2c3280-9f4a-4a33-a496-d06f0ba6c53d admin admin] PATH_INFO: `/v3/domains` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:28.844718 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2c3280-9f4a-4a33-a496-d06f0ba6c53d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:58:28.845152 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2c3280-9f4a-4a33-a496-d06f0ba6c53d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:58:28.848550 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9e4b99-9fe6-4314-88e4-cb6a1a6f7097 admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:58:28.849032 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2c3280-9f4a-4a33-a496-d06f0ba6c53d admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:58:28.861857 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddce92da-9afd-48d2-817a-6dc1998007ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:58:28.862533 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddce92da-9afd-48d2-817a-6dc1998007ea admin admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:28.865465 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddce92da-9afd-48d2-817a-6dc1998007ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:28.865913 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddce92da-9afd-48d2-817a-6dc1998007ea admin admin] PATH_INFO: `/v3/roles` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:28.866625 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddce92da-9afd-48d2-817a-6dc1998007ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:58:28.872670 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddce92da-9afd-48d2-817a-6dc1998007ea admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:58:28.877532 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 851/1685] 10.222.0.33 () {58 vars in 1153 bytes} [Thu Sep 1 14:58:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:58:28.880366 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2c3280-9f4a-4a33-a496-d06f0ba6c53d admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:58:28.892402 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2c3280-9f4a-4a33-a496-d06f0ba6c53d admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:58:28.902727 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e237b5a2-8af3-4347-82fc-d7261070eab6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:58:28.904250 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e237b5a2-8af3-4347-82fc-d7261070eab6 admin admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:28.904749 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e237b5a2-8af3-4347-82fc-d7261070eab6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:28.905226 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e237b5a2-8af3-4347-82fc-d7261070eab6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:28.906921 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e237b5a2-8af3-4347-82fc-d7261070eab6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:58:28.908843 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e237b5a2-8af3-4347-82fc-d7261070eab6 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:58:28.912285 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 835/1686] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 1 14:58:28 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:58:28.915885 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddce92da-9afd-48d2-817a-6dc1998007ea admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:58:28.923602 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 852/1687] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 1 14:58:28 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:58:28.925219 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e237b5a2-8af3-4347-82fc-d7261070eab6 admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:58:28.941458 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddce92da-9afd-48d2-817a-6dc1998007ea admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:58:28.944316 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e237b5a2-8af3-4347-82fc-d7261070eab6 admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:58:28.959640 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace4d5be-432d-4c6d-97e3-c5500d1a4223 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:58:28.960453 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee5608e-e045-41ee-9d33-e152e1e452b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:58:28.961222 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace4d5be-432d-4c6d-97e3-c5500d1a4223 admin admin] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:28.962021 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee5608e-e045-41ee-9d33-e152e1e452b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:28.962429 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace4d5be-432d-4c6d-97e3-c5500d1a4223 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:28.962815 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee5608e-e045-41ee-9d33-e152e1e452b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:28.963191 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee5608e-e045-41ee-9d33-e152e1e452b3 admin admin] PATH_INFO: `/v3/projects/470ea9bada8a49c9840e745de1c91f28/users/53a257a41bbb4888901442650d4ba5a7/roles/5d1feef0e13d4874928bd86258c4e3b2` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:28.963563 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace4d5be-432d-4c6d-97e3-c5500d1a4223 admin admin] PATH_INFO: `/v3/projects` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:28.965078 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace4d5be-432d-4c6d-97e3-c5500d1a4223 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:58:28.969287 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace4d5be-432d-4c6d-97e3-c5500d1a4223 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:58:29.002900 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee5608e-e045-41ee-9d33-e152e1e452b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=470ea9bada8a49c9840e745de1c91f28, user_id=53a257a41bbb4888901442650d4ba5a7, role_id=5d1feef0e13d4874928bd86258c4e3b2)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:58:29.010431 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee5608e-e045-41ee-9d33-e152e1e452b3 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:58:29.064310 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 836/1688] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Sep 1 14:58:28 2022] POST /identity/v3/projects => generated 363 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:58:29.065484 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace4d5be-432d-4c6d-97e3-c5500d1a4223 admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:58:29.074271 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 853/1689] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 1 14:58:28 2022] PUT /identity/v3/projects/470ea9bada8a49c9840e745de1c91f28/users/53a257a41bbb4888901442650d4ba5a7/roles/5d1feef0e13d4874928bd86258c4e3b2 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:58:29.076223 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee5608e-e045-41ee-9d33-e152e1e452b3 admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:58:29.233850 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace4d5be-432d-4c6d-97e3-c5500d1a4223 admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:58:29.246322 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee5608e-e045-41ee-9d33-e152e1e452b3 admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:58:29.249943 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c98ae1-94b5-4b07-9d58-7303438bdd64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:58:29.250755 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c98ae1-94b5-4b07-9d58-7303438bdd64 admin admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:29.251147 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c98ae1-94b5-4b07-9d58-7303438bdd64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:29.251685 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c98ae1-94b5-4b07-9d58-7303438bdd64 admin admin] PATH_INFO: `/v3/roles` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:29.254108 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c98ae1-94b5-4b07-9d58-7303438bdd64 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:58:29.259039 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf69217c-0b62-47c0-8ca1-7d3654690044 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:58:29.259668 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf69217c-0b62-47c0-8ca1-7d3654690044 admin admin] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:29.259947 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf69217c-0b62-47c0-8ca1-7d3654690044 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:29.260218 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf69217c-0b62-47c0-8ca1-7d3654690044 admin admin] PATH_INFO: `/v3/users` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:29.261191 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf69217c-0b62-47c0-8ca1-7d3654690044 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:58:29.262405 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c98ae1-94b5-4b07-9d58-7303438bdd64 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:58:29.263191 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf69217c-0b62-47c0-8ca1-7d3654690044 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:58:29.290301 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 837/1690] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 1 14:58:29 2022] GET /identity/v3/roles => generated 1420 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:58:29.290301 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb42287-a7fb-4b8e-a637-d27918a0515d None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:29.291001 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb42287-a7fb-4b8e-a637-d27918a0515d None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:29.291658 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb42287-a7fb-4b8e-a637-d27918a0515d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:29.303291 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:58:29.307755 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 854/1691] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Sep 1 14:58:29 2022] POST /identity/v3/users => generated 282 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:58:29.308778 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf69217c-0b62-47c0-8ca1-7d3654690044 admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:58:29.332345 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf69217c-0b62-47c0-8ca1-7d3654690044 admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:58:29.344790 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fd49a8-4242-4a72-a9f0-a19ee0d0e173 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:58:29.345479 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fd49a8-4242-4a72-a9f0-a19ee0d0e173 admin admin] REQUEST_METHOD: `PUT` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:29.346265 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fd49a8-4242-4a72-a9f0-a19ee0d0e173 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:29.346265 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fd49a8-4242-4a72-a9f0-a19ee0d0e173 admin admin] PATH_INFO: `/v3/projects/5fdf47652b8942bf8e80c8d229ca1896/users/cdd15ae9e0944062bfbe8acb18433b9f/roles/5d1feef0e13d4874928bd86258c4e3b2` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:29.362120 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-6fb42287-a7fb-4b8e-a637-d27918a0515d None None] MFA Rules not processed for user `53a257a41bbb4888901442650d4ba5a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:58:29.363428 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-6fb42287-a7fb-4b8e-a637-d27918a0515d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:58:29.380920 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fd49a8-4242-4a72-a9f0-a19ee0d0e173 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5fdf47652b8942bf8e80c8d229ca1896, user_id=cdd15ae9e0944062bfbe8acb18433b9f, role_id=5d1feef0e13d4874928bd86258c4e3b2)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:58:29.384022 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fd49a8-4242-4a72-a9f0-a19ee0d0e173 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:58:29.418662 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 855/1692] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 1 14:58:29 2022] PUT /identity/v3/projects/5fdf47652b8942bf8e80c8d229ca1896/users/cdd15ae9e0944062bfbe8acb18433b9f/roles/5d1feef0e13d4874928bd86258c4e3b2 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:58:29.418662 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fd49a8-4242-4a72-a9f0-a19ee0d0e173 admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:58:29.486154 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:58:29.565001 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fd49a8-4242-4a72-a9f0-a19ee0d0e173 admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:58:29.566317 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 838/1693] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 14:58:29 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 278 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:58:29.569745 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1297d394-970d-4114-861b-f48ab2596755 None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:29.569745 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1297d394-970d-4114-861b-f48ab2596755 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:29.571416 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1297d394-970d-4114-861b-f48ab2596755 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:29.579693 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6762591d-9278-4bb2-8e31-8b0cb34e7ea5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:58:29.581936 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6762591d-9278-4bb2-8e31-8b0cb34e7ea5 admin admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:29.582361 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6762591d-9278-4bb2-8e31-8b0cb34e7ea5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:29.582681 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6762591d-9278-4bb2-8e31-8b0cb34e7ea5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:29.583676 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6762591d-9278-4bb2-8e31-8b0cb34e7ea5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:58:29.586478 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6762591d-9278-4bb2-8e31-8b0cb34e7ea5 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:58:29.620557 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 856/1694] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 1 14:58:29 2022] GET /identity/v3/roles => generated 1420 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:58:29.622780 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6762591d-9278-4bb2-8e31-8b0cb34e7ea5 admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:58:29.628209 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-6762591d-9278-4bb2-8e31-8b0cb34e7ea5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:58:29.671725 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-1297d394-970d-4114-861b-f48ab2596755 None None] MFA Rules not processed for user `cdd15ae9e0944062bfbe8acb18433b9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:58:29.673086 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-1297d394-970d-4114-861b-f48ab2596755 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:58:29.836077 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6762591d-9278-4bb2-8e31-8b0cb34e7ea5 admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:58:29.841500 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574187f8-d60b-41db-ad58-95182ac7fe9d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3069f72ce98049b99c9d98229a62bb51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:58:29.842130 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574187f8-d60b-41db-ad58-95182ac7fe9d service glance] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:29.842479 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574187f8-d60b-41db-ad58-95182ac7fe9d service glance] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:29.842757 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574187f8-d60b-41db-ad58-95182ac7fe9d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:29.862862 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 839/1695] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 14:58:29 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 296 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:58:29.865043 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1297d394-970d-4114-861b-f48ab2596755 None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:58:29.902422 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1297d394-970d-4114-861b-f48ab2596755 None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:58:29.916872 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df35664-ee2a-40af-ad80-e9726fd883bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:58:29.918510 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df35664-ee2a-40af-ad80-e9726fd883bb admin admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:29.919198 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df35664-ee2a-40af-ad80-e9726fd883bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:29.920298 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df35664-ee2a-40af-ad80-e9726fd883bb admin admin] PATH_INFO: `/v3/roles` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:29.921082 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df35664-ee2a-40af-ad80-e9726fd883bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:58:29.925800 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df35664-ee2a-40af-ad80-e9726fd883bb admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:58:29.954360 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 840/1696] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 1 14:58:29 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:58:29.954956 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df35664-ee2a-40af-ad80-e9726fd883bb admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:58:29.957329 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-5df35664-ee2a-40af-ad80-e9726fd883bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:58:29.985144 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df35664-ee2a-40af-ad80-e9726fd883bb admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:58:30.007193 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74abdcf0-d31d-4cab-976a-6e43d51137f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:58:30.009949 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74abdcf0-d31d-4cab-976a-6e43d51137f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:30.010416 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574187f8-d60b-41db-ad58-95182ac7fe9d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:58:30.010979 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74abdcf0-d31d-4cab-976a-6e43d51137f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:30.011227 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74abdcf0-d31d-4cab-976a-6e43d51137f6 admin admin] PATH_INFO: `/v3/projects/2e452af76635415ebc7b06a4b9f625e6` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:30.012321 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574187f8-d60b-41db-ad58-95182ac7fe9d service glance] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:58:30.014994 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74abdcf0-d31d-4cab-976a-6e43d51137f6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2e452af76635415ebc7b06a4b9f625e6)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:58:30.018181 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74abdcf0-d31d-4cab-976a-6e43d51137f6 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:58:30.041813 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 857/1697] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 14:58:29 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:58:30.045209 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574187f8-d60b-41db-ad58-95182ac7fe9d service glance] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:58:30.066502 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574187f8-d60b-41db-ad58-95182ac7fe9d service glance] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:58:30.083215 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cf437c-7395-40d4-8bb9-1af0a438a997 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:58:30.083924 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cf437c-7395-40d4-8bb9-1af0a438a997 admin admin] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:30.084208 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cf437c-7395-40d4-8bb9-1af0a438a997 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:30.084508 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cf437c-7395-40d4-8bb9-1af0a438a997 admin admin] PATH_INFO: `/v3/projects` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:30.085252 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cf437c-7395-40d4-8bb9-1af0a438a997 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:58:30.087408 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cf437c-7395-40d4-8bb9-1af0a438a997 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:58:30.137376 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-74abdcf0-d31d-4cab-976a-6e43d51137f6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2e452af76635415ebc7b06a4b9f625e6', 'request_id': 'req-74abdcf0-d31d-4cab-976a-6e43d51137f6'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:58:30.149510 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-74abdcf0-d31d-4cab-976a-6e43d51137f6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2e452af76635415ebc7b06a4b9f625e6', 'request_id': 'req-74abdcf0-d31d-4cab-976a-6e43d51137f6'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:58:30.158247 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 858/1698] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Sep 1 14:58:30 2022] POST /identity/v3/projects => generated 371 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:58:30.159296 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cf437c-7395-40d4-8bb9-1af0a438a997 admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:58:30.168717 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-74abdcf0-d31d-4cab-976a-6e43d51137f6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2e452af76635415ebc7b06a4b9f625e6', 'request_id': 'req-74abdcf0-d31d-4cab-976a-6e43d51137f6'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:58:30.184524 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:58:30.204586 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-74abdcf0-d31d-4cab-976a-6e43d51137f6 admin admin] The token cache is being invalidate because project 2e452af76635415ebc7b06a4b9f625e6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81341) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:58:30.204940 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-74abdcf0-d31d-4cab-976a-6e43d51137f6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:58:30.206546 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-74abdcf0-d31d-4cab-976a-6e43d51137f6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:58:30.211212 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 841/1699] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Sep 1 14:58:29 2022] DELETE /identity/v3/projects/2e452af76635415ebc7b06a4b9f625e6 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:58:30.212260 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74abdcf0-d31d-4cab-976a-6e43d51137f6 admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:58:30.218528 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-74abdcf0-d31d-4cab-976a-6e43d51137f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:58:30.243257 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:58:30.298696 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cf437c-7395-40d4-8bb9-1af0a438a997 admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:58:30.305431 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25953756-1a14-439d-b72f-5aae6ca88884 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:58:30.306474 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25953756-1a14-439d-b72f-5aae6ca88884 admin admin] REQUEST_METHOD: `PUT` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:30.306916 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25953756-1a14-439d-b72f-5aae6ca88884 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:30.307332 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25953756-1a14-439d-b72f-5aae6ca88884 admin admin] PATH_INFO: `/v3/projects/055f989992444be394ef45c6fdbb6a53/users/a38ba4fe45864fe880ea0f77a14f5591/roles/5d1feef0e13d4874928bd86258c4e3b2` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:30.325673 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25953756-1a14-439d-b72f-5aae6ca88884 admin admin] RBAC: Authorizing `identity:create_grant(project_id=055f989992444be394ef45c6fdbb6a53, user_id=a38ba4fe45864fe880ea0f77a14f5591, role_id=5d1feef0e13d4874928bd86258c4e3b2)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:58:30.328746 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25953756-1a14-439d-b72f-5aae6ca88884 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:58:30.357307 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 859/1700] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 1 14:58:30 2022] PUT /identity/v3/projects/055f989992444be394ef45c6fdbb6a53/users/a38ba4fe45864fe880ea0f77a14f5591/roles/5d1feef0e13d4874928bd86258c4e3b2 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:58:30.360204 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18607132-fc98-4098-a203-d616f2a68cae None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:30.361955 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18607132-fc98-4098-a203-d616f2a68cae None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:30.362416 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18607132-fc98-4098-a203-d616f2a68cae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:30.391090 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74abdcf0-d31d-4cab-976a-6e43d51137f6 admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:58:30.404256 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-18607132-fc98-4098-a203-d616f2a68cae None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:58:30.407002 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-18607132-fc98-4098-a203-d616f2a68cae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:58:30.546630 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457913fd-0063-4d85-b8b1-00d7540b56f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:58:30.547263 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457913fd-0063-4d85-b8b1-00d7540b56f7 admin admin] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:30.547544 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457913fd-0063-4d85-b8b1-00d7540b56f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:30.548343 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457913fd-0063-4d85-b8b1-00d7540b56f7 admin admin] PATH_INFO: `/v3/users` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:30.548448 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457913fd-0063-4d85-b8b1-00d7540b56f7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:58:30.550667 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457913fd-0063-4d85-b8b1-00d7540b56f7 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:58:30.573510 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 860/1701] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 14:58:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:58:30.574732 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91790575-41ae-402b-8bc9-ced5daf10554 None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:30.575135 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91790575-41ae-402b-8bc9-ced5daf10554 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:30.575347 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91790575-41ae-402b-8bc9-ced5daf10554 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:30.602176 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 842/1702] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Sep 1 14:58:30 2022] POST /identity/v3/users => generated 292 bytes in 390 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:58:30.603772 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbf7abd-f0ae-4fa5-b32e-0df721692262 None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:30.604167 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbf7abd-f0ae-4fa5-b32e-0df721692262 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:30.604936 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbf7abd-f0ae-4fa5-b32e-0df721692262 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:30.639013 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-bfbf7abd-f0ae-4fa5-b32e-0df721692262 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:58:30.640179 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-bfbf7abd-f0ae-4fa5-b32e-0df721692262 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:58:30.646574 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-91790575-41ae-402b-8bc9-ced5daf10554 None None] MFA Rules not processed for user `a38ba4fe45864fe880ea0f77a14f5591`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:58:30.647803 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-91790575-41ae-402b-8bc9-ced5daf10554 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:58:30.669216 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 843/1703] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 14:58:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:58:30.669848 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbf7abd-f0ae-4fa5-b32e-0df721692262 None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:58:30.697562 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbf7abd-f0ae-4fa5-b32e-0df721692262 None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:58:30.712010 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a1efb7-3321-46a2-9453-f4dec5105622 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:58:30.712636 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a1efb7-3321-46a2-9453-f4dec5105622 admin admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:30.712919 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a1efb7-3321-46a2-9453-f4dec5105622 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:30.713193 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a1efb7-3321-46a2-9453-f4dec5105622 admin admin] PATH_INFO: `/v3/roles` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:30.714303 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a1efb7-3321-46a2-9453-f4dec5105622 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:58:30.716389 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a1efb7-3321-46a2-9453-f4dec5105622 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:58:30.748606 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 844/1704] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 1 14:58:30 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:58:30.749585 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a1efb7-3321-46a2-9453-f4dec5105622 admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:58:30.772698 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a1efb7-3321-46a2-9453-f4dec5105622 admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:58:30.786977 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba111cb-75c0-4efd-8bb9-96f8253eac5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:58:30.787601 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba111cb-75c0-4efd-8bb9-96f8253eac5e admin admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:30.787885 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba111cb-75c0-4efd-8bb9-96f8253eac5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:30.788774 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba111cb-75c0-4efd-8bb9-96f8253eac5e admin admin] PATH_INFO: `/v3/domains` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:30.789927 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba111cb-75c0-4efd-8bb9-96f8253eac5e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81341) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:58:30.790253 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba111cb-75c0-4efd-8bb9-96f8253eac5e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:58:30.791903 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba111cb-75c0-4efd-8bb9-96f8253eac5e admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:58:30.816881 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 861/1705] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 14:58:30 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:58:30.817953 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91790575-41ae-402b-8bc9-ced5daf10554 None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:58:30.827429 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 845/1706] 10.222.0.33 () {58 vars in 1153 bytes} [Thu Sep 1 14:58:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:58:30.828340 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba111cb-75c0-4efd-8bb9-96f8253eac5e admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:58:30.831541 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-6ba111cb-75c0-4efd-8bb9-96f8253eac5e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:58:30.842379 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91790575-41ae-402b-8bc9-ced5daf10554 None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:58:30.857549 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f15425-6e7b-426b-9f99-c1e2f44423e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:58:30.858278 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f15425-6e7b-426b-9f99-c1e2f44423e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:30.858549 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f15425-6e7b-426b-9f99-c1e2f44423e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:30.858829 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f15425-6e7b-426b-9f99-c1e2f44423e1 admin admin] PATH_INFO: `/v3/projects/5e984423596a4cffa83e0b537e72ac88/users/d4384b5f37614a23b12309a063d6d40a/roles/2a2adb769e55476b82e5014897e0a9c0` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:30.867520 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba111cb-75c0-4efd-8bb9-96f8253eac5e admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:58:30.881158 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016bcf60-ce4c-484c-b59c-98eb96e6eed0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:58:30.881953 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016bcf60-ce4c-484c-b59c-98eb96e6eed0 admin admin] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:30.882257 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016bcf60-ce4c-484c-b59c-98eb96e6eed0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:30.882535 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016bcf60-ce4c-484c-b59c-98eb96e6eed0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:30.883319 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016bcf60-ce4c-484c-b59c-98eb96e6eed0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:58:30.888140 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016bcf60-ce4c-484c-b59c-98eb96e6eed0 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:58:30.913333 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f15425-6e7b-426b-9f99-c1e2f44423e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e984423596a4cffa83e0b537e72ac88, user_id=d4384b5f37614a23b12309a063d6d40a, role_id=2a2adb769e55476b82e5014897e0a9c0)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:58:30.920593 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f15425-6e7b-426b-9f99-c1e2f44423e1 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:58:30.960665 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 862/1707] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 1 14:58:30 2022] PUT /identity/v3/projects/5e984423596a4cffa83e0b537e72ac88/users/d4384b5f37614a23b12309a063d6d40a/roles/2a2adb769e55476b82e5014897e0a9c0 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:58:30.964155 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f15425-6e7b-426b-9f99-c1e2f44423e1 admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:58:30.978618 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 846/1708] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Sep 1 14:58:30 2022] POST /identity/v3/projects => generated 363 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:58:30.980076 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016bcf60-ce4c-484c-b59c-98eb96e6eed0 admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:58:31.039117 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:58:31.100434 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:58:31.133566 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f15425-6e7b-426b-9f99-c1e2f44423e1 admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:58:31.148048 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43dc0e43-eb95-461b-b657-3eb946546452 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:58:31.148682 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dc0e43-eb95-461b-b657-3eb946546452 admin admin] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:31.149033 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dc0e43-eb95-461b-b657-3eb946546452 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:31.149307 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dc0e43-eb95-461b-b657-3eb946546452 admin admin] PATH_INFO: `/v3/projects` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:31.149962 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43dc0e43-eb95-461b-b657-3eb946546452 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:58:31.154385 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43dc0e43-eb95-461b-b657-3eb946546452 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:58:31.158224 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016bcf60-ce4c-484c-b59c-98eb96e6eed0 admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:58:31.176231 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069cfd04-6693-4a00-9c8a-6b7c1cb990c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:58:31.178368 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069cfd04-6693-4a00-9c8a-6b7c1cb990c5 admin admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:31.180175 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069cfd04-6693-4a00-9c8a-6b7c1cb990c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:31.180570 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069cfd04-6693-4a00-9c8a-6b7c1cb990c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:31.184074 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069cfd04-6693-4a00-9c8a-6b7c1cb990c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:58:31.189603 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069cfd04-6693-4a00-9c8a-6b7c1cb990c5 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:58:31.212292 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 847/1709] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 1 14:58:30 2022] GET /identity/v3/roles => generated 1420 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:58:31.214147 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069cfd04-6693-4a00-9c8a-6b7c1cb990c5 admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:58:31.244729 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 863/1710] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Sep 1 14:58:30 2022] POST /identity/v3/projects => generated 381 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:58:31.244729 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43dc0e43-eb95-461b-b657-3eb946546452 admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:58:31.245849 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069cfd04-6693-4a00-9c8a-6b7c1cb990c5 admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:58:31.409581 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63649218-cd13-4dec-8068-ad93ffeb062b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:58:31.410476 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63649218-cd13-4dec-8068-ad93ffeb062b admin admin] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:31.410476 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63649218-cd13-4dec-8068-ad93ffeb062b admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:31.410778 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63649218-cd13-4dec-8068-ad93ffeb062b admin admin] PATH_INFO: `/v3/users` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:31.412380 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63649218-cd13-4dec-8068-ad93ffeb062b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:58:31.414269 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63649218-cd13-4dec-8068-ad93ffeb062b admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:58:31.415419 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43dc0e43-eb95-461b-b657-3eb946546452 admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:58:31.444132 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7591bfdb-405b-4b67-959f-9e8e22e34fe2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:58:31.444499 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7591bfdb-405b-4b67-959f-9e8e22e34fe2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:31.444786 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7591bfdb-405b-4b67-959f-9e8e22e34fe2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:31.445072 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7591bfdb-405b-4b67-959f-9e8e22e34fe2 admin admin] PATH_INFO: `/v3/projects/5e984423596a4cffa83e0b537e72ac88/users/d4384b5f37614a23b12309a063d6d40a/roles/2a2adb769e55476b82e5014897e0a9c0` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:31.451721 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7591bfdb-405b-4b67-959f-9e8e22e34fe2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e984423596a4cffa83e0b537e72ac88, user_id=d4384b5f37614a23b12309a063d6d40a, role_id=2a2adb769e55476b82e5014897e0a9c0)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:58:31.458712 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7591bfdb-405b-4b67-959f-9e8e22e34fe2 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:58:31.466495 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 848/1711] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Sep 1 14:58:31 2022] POST /identity/v3/users => generated 288 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:58:31.467597 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63649218-cd13-4dec-8068-ad93ffeb062b admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:58:31.487837 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63649218-cd13-4dec-8068-ad93ffeb062b admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:58:31.503794 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 864/1712] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 1 14:58:31 2022] PUT /identity/v3/projects/5e984423596a4cffa83e0b537e72ac88/users/d4384b5f37614a23b12309a063d6d40a/roles/2a2adb769e55476b82e5014897e0a9c0 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:58:31.504895 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7591bfdb-405b-4b67-959f-9e8e22e34fe2 admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:58:31.621361 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545b5e53-77a8-4d7e-b07a-a161d7cf26bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:58:31.621578 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545b5e53-77a8-4d7e-b07a-a161d7cf26bf admin admin] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:31.621857 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545b5e53-77a8-4d7e-b07a-a161d7cf26bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:31.622133 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545b5e53-77a8-4d7e-b07a-a161d7cf26bf admin admin] PATH_INFO: `/v3/users` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:31.637402 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545b5e53-77a8-4d7e-b07a-a161d7cf26bf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:58:31.637402 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545b5e53-77a8-4d7e-b07a-a161d7cf26bf admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:58:31.709542 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7591bfdb-405b-4b67-959f-9e8e22e34fe2 admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:58:31.721783 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 849/1713] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Sep 1 14:58:31 2022] POST /identity/v3/users => generated 291 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:58:31.725950 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d547e526-a25b-4e4f-8d1f-83874dc1d76d None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:31.726272 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d547e526-a25b-4e4f-8d1f-83874dc1d76d None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:31.726594 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d547e526-a25b-4e4f-8d1f-83874dc1d76d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:31.731230 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0369e211-58b5-4b1a-a9f3-d3543368201d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:58:31.732057 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0369e211-58b5-4b1a-a9f3-d3543368201d admin admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:31.732275 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0369e211-58b5-4b1a-a9f3-d3543368201d admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:31.732443 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0369e211-58b5-4b1a-a9f3-d3543368201d admin admin] PATH_INFO: `/v3/roles` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:31.733056 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0369e211-58b5-4b1a-a9f3-d3543368201d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:58:31.734726 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0369e211-58b5-4b1a-a9f3-d3543368201d admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:58:31.773430 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 865/1714] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 1 14:58:31 2022] GET /identity/v3/roles => generated 1420 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:58:31.777744 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0369e211-58b5-4b1a-a9f3-d3543368201d admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:58:31.809716 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0369e211-58b5-4b1a-a9f3-d3543368201d admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:58:31.815447 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-d547e526-a25b-4e4f-8d1f-83874dc1d76d None None] MFA Rules not processed for user `d4384b5f37614a23b12309a063d6d40a`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:58:31.817083 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-d547e526-a25b-4e4f-8d1f-83874dc1d76d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:58:31.827204 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa292ca9-1c62-4749-aee0-bc32bac6eb9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:58:31.827837 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa292ca9-1c62-4749-aee0-bc32bac6eb9f admin admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:31.834701 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa292ca9-1c62-4749-aee0-bc32bac6eb9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:31.835033 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa292ca9-1c62-4749-aee0-bc32bac6eb9f admin admin] PATH_INFO: `/v3/roles` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:31.835741 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa292ca9-1c62-4749-aee0-bc32bac6eb9f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:58:31.837390 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa292ca9-1c62-4749-aee0-bc32bac6eb9f admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:58:31.878727 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 866/1715] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 1 14:58:31 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:58:31.879902 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa292ca9-1c62-4749-aee0-bc32bac6eb9f admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:58:31.921296 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa292ca9-1c62-4749-aee0-bc32bac6eb9f admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:58:31.924435 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:58:31.950626 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b54e1a-0694-4b58-b759-7fa1e81ca6e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:58:31.951983 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b54e1a-0694-4b58-b759-7fa1e81ca6e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:31.952497 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b54e1a-0694-4b58-b759-7fa1e81ca6e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:31.953091 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b54e1a-0694-4b58-b759-7fa1e81ca6e9 admin admin] PATH_INFO: `/v3/projects/298db1d1c9164f93af26f4ded1819ce1/users/d96325e5eace4e8799cd40dcfd8b3cb0/roles/2a2adb769e55476b82e5014897e0a9c0` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:32.000165 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b54e1a-0694-4b58-b759-7fa1e81ca6e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=298db1d1c9164f93af26f4ded1819ce1, user_id=d96325e5eace4e8799cd40dcfd8b3cb0, role_id=2a2adb769e55476b82e5014897e0a9c0)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:58:32.007771 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 850/1716] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 14:58:31 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:58:32.020970 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b54e1a-0694-4b58-b759-7fa1e81ca6e9 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:58:32.023123 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d547e526-a25b-4e4f-8d1f-83874dc1d76d None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:58:32.082670 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 867/1717] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 1 14:58:31 2022] PUT /identity/v3/projects/298db1d1c9164f93af26f4ded1819ce1/users/d96325e5eace4e8799cd40dcfd8b3cb0/roles/2a2adb769e55476b82e5014897e0a9c0 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:58:32.082670 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b54e1a-0694-4b58-b759-7fa1e81ca6e9 admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:58:32.086006 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d547e526-a25b-4e4f-8d1f-83874dc1d76d None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:58:32.093202 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-72b54e1a-0694-4b58-b759-7fa1e81ca6e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:58:32.100441 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:58:32.231407 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b54e1a-0694-4b58-b759-7fa1e81ca6e9 admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:58:32.236380 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d805677-c335-4776-a11c-4d3df0f99f7f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3b81739e34e4e648f88b6d73fcb5e46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:58:32.237017 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d805677-c335-4776-a11c-4d3df0f99f7f service cinder] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:32.237300 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d805677-c335-4776-a11c-4d3df0f99f7f service cinder] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:32.237685 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d805677-c335-4776-a11c-4d3df0f99f7f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:32.242154 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6da5b4-5f44-4c67-944f-824bbbd2ac05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:58:32.243171 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6da5b4-5f44-4c67-944f-824bbbd2ac05 admin admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:32.243603 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6da5b4-5f44-4c67-944f-824bbbd2ac05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:32.244031 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6da5b4-5f44-4c67-944f-824bbbd2ac05 admin admin] PATH_INFO: `/v3/roles` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:32.245030 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6da5b4-5f44-4c67-944f-824bbbd2ac05 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:58:32.247009 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6da5b4-5f44-4c67-944f-824bbbd2ac05 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:58:32.265278 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 851/1718] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 1 14:58:32 2022] GET /identity/v3/roles => generated 1420 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:58:32.266518 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6da5b4-5f44-4c67-944f-824bbbd2ac05 admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:58:32.280765 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6da5b4-5f44-4c67-944f-824bbbd2ac05 admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:58:32.290487 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99765885-1bb5-4009-b9bc-7032e5c829d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:58:32.291618 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99765885-1bb5-4009-b9bc-7032e5c829d8 admin admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:32.292060 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99765885-1bb5-4009-b9bc-7032e5c829d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:32.292527 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99765885-1bb5-4009-b9bc-7032e5c829d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:32.293383 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99765885-1bb5-4009-b9bc-7032e5c829d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:58:32.295390 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99765885-1bb5-4009-b9bc-7032e5c829d8 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:58:32.312935 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 852/1719] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 1 14:58:32 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:58:32.314279 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99765885-1bb5-4009-b9bc-7032e5c829d8 admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:58:32.326986 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99765885-1bb5-4009-b9bc-7032e5c829d8 admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:58:32.333753 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a587a8-86b4-48fe-94bf-346e11a6d087 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:58:32.335311 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a587a8-86b4-48fe-94bf-346e11a6d087 admin admin] REQUEST_METHOD: `PUT` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:32.335886 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a587a8-86b4-48fe-94bf-346e11a6d087 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:32.336314 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a587a8-86b4-48fe-94bf-346e11a6d087 admin admin] PATH_INFO: `/v3/projects/a59b1cce594f41f8a7e1f4f36e8f3386/users/b8d124cf913b46babd8a9d0a528ee3b0/roles/5d1feef0e13d4874928bd86258c4e3b2` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:32.364538 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a587a8-86b4-48fe-94bf-346e11a6d087 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a59b1cce594f41f8a7e1f4f36e8f3386, user_id=b8d124cf913b46babd8a9d0a528ee3b0, role_id=5d1feef0e13d4874928bd86258c4e3b2)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:58:32.368853 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a587a8-86b4-48fe-94bf-346e11a6d087 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:58:32.405463 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 853/1720] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 1 14:58:32 2022] PUT /identity/v3/projects/a59b1cce594f41f8a7e1f4f36e8f3386/users/b8d124cf913b46babd8a9d0a528ee3b0/roles/5d1feef0e13d4874928bd86258c4e3b2 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:58:32.406634 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a587a8-86b4-48fe-94bf-346e11a6d087 admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:58:32.409364 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d805677-c335-4776-a11c-4d3df0f99f7f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:58:32.412559 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d805677-c335-4776-a11c-4d3df0f99f7f service cinder] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:58:32.432458 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 868/1721] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 14:58:32 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:58:32.433816 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cd1ccc-63a2-4d05-8d98-2bfcdf0c8e00 None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:32.434111 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cd1ccc-63a2-4d05-8d98-2bfcdf0c8e00 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:32.434417 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cd1ccc-63a2-4d05-8d98-2bfcdf0c8e00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:32.507894 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-79cd1ccc-63a2-4d05-8d98-2bfcdf0c8e00 None None] MFA Rules not processed for user `b8d124cf913b46babd8a9d0a528ee3b0`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:58:32.509309 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-79cd1ccc-63a2-4d05-8d98-2bfcdf0c8e00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:58:32.555783 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a587a8-86b4-48fe-94bf-346e11a6d087 admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:58:32.567498 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a489a8ad-1dc2-4597-9e06-dbf8ed9d626a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:58:32.568418 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a489a8ad-1dc2-4597-9e06-dbf8ed9d626a admin admin] REQUEST_METHOD: `PUT` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:32.568842 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a489a8ad-1dc2-4597-9e06-dbf8ed9d626a admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:32.569273 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a489a8ad-1dc2-4597-9e06-dbf8ed9d626a admin admin] PATH_INFO: `/v3/projects/298db1d1c9164f93af26f4ded1819ce1/users/d96325e5eace4e8799cd40dcfd8b3cb0/roles/2a2adb769e55476b82e5014897e0a9c0` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:32.572700 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a489a8ad-1dc2-4597-9e06-dbf8ed9d626a admin admin] RBAC: Authorizing `identity:create_grant(project_id=298db1d1c9164f93af26f4ded1819ce1, user_id=d96325e5eace4e8799cd40dcfd8b3cb0, role_id=2a2adb769e55476b82e5014897e0a9c0)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:58:32.585173 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a489a8ad-1dc2-4597-9e06-dbf8ed9d626a admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:58:32.629048 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 854/1722] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 1 14:58:32 2022] PUT /identity/v3/projects/298db1d1c9164f93af26f4ded1819ce1/users/d96325e5eace4e8799cd40dcfd8b3cb0/roles/2a2adb769e55476b82e5014897e0a9c0 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:58:32.629048 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a489a8ad-1dc2-4597-9e06-dbf8ed9d626a admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:58:32.713812 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:58:32.730113 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 869/1723] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 14:58:32 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 297 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:58:32.732244 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14364872-370c-4735-b244-ca1a6bae343c None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:32.752135 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a489a8ad-1dc2-4597-9e06-dbf8ed9d626a admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:58:32.762286 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560cba29-8619-470d-9d27-1d14b2d246c6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3b81739e34e4e648f88b6d73fcb5e46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:58:32.762975 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14364872-370c-4735-b244-ca1a6bae343c None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:32.763095 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560cba29-8619-470d-9d27-1d14b2d246c6 service cinder] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:32.763258 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560cba29-8619-470d-9d27-1d14b2d246c6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:32.763368 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14364872-370c-4735-b244-ca1a6bae343c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:32.763538 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560cba29-8619-470d-9d27-1d14b2d246c6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:32.773964 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-560cba29-8619-470d-9d27-1d14b2d246c6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:58:32.867422 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-14364872-370c-4735-b244-ca1a6bae343c None None] MFA Rules not processed for user `d96325e5eace4e8799cd40dcfd8b3cb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:58:32.869240 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-14364872-370c-4735-b244-ca1a6bae343c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:58:32.900153 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560cba29-8619-470d-9d27-1d14b2d246c6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:58:32.901630 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560cba29-8619-470d-9d27-1d14b2d246c6 service cinder] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:58:32.921419 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 855/1724] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 14:58:32 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:58:32.922314 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560cba29-8619-470d-9d27-1d14b2d246c6 service cinder] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:58:32.942669 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:58:33.047596 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 870/1725] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 14:58:32 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 316 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:58:33.058932 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0102b9c5-67ce-473d-a4da-1d5ed392de48 None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:33.060210 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0102b9c5-67ce-473d-a4da-1d5ed392de48 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:33.060596 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0102b9c5-67ce-473d-a4da-1d5ed392de48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:33.073097 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560cba29-8619-470d-9d27-1d14b2d246c6 service cinder] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:58:33.086591 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8e8723-9ae4-400a-9248-0770462955d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:58:33.086591 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8e8723-9ae4-400a-9248-0770462955d8 admin admin] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:33.086939 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8e8723-9ae4-400a-9248-0770462955d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:33.087137 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8e8723-9ae4-400a-9248-0770462955d8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:33.087892 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8e8723-9ae4-400a-9248-0770462955d8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:58:33.090103 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8e8723-9ae4-400a-9248-0770462955d8 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:58:33.153184 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 856/1726] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Sep 1 14:58:32 2022] POST /identity/v3/projects => generated 379 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:58:33.153883 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8e8723-9ae4-400a-9248-0770462955d8 admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:58:33.155970 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-8f8e8723-9ae4-400a-9248-0770462955d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:58:33.185170 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-0102b9c5-67ce-473d-a4da-1d5ed392de48 None None] MFA Rules not processed for user `24b6f275ffb04ffeae3c2097bfd040f5`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:58:33.191645 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-0102b9c5-67ce-473d-a4da-1d5ed392de48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:58:33.290144 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8e8723-9ae4-400a-9248-0770462955d8 admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:58:33.296152 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55c2ed1-fe90-49a2-b48a-cfef754b08b9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3b81739e34e4e648f88b6d73fcb5e46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:58:33.296826 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55c2ed1-fe90-49a2-b48a-cfef754b08b9 service cinder] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:33.297158 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55c2ed1-fe90-49a2-b48a-cfef754b08b9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:33.315921 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55c2ed1-fe90-49a2-b48a-cfef754b08b9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:33.332223 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 871/1727] 10.222.0.33 () {58 vars in 1009 bytes} [Thu Sep 1 14:58:33 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 284 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:58:33.333873 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0102b9c5-67ce-473d-a4da-1d5ed392de48 None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:58:33.447231 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:58:33.491276 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0102b9c5-67ce-473d-a4da-1d5ed392de48 None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:58:33.505404 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8190cdaf-43dd-4eae-9126-17e6bdc27221 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:58:33.507072 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8190cdaf-43dd-4eae-9126-17e6bdc27221 admin admin] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:33.507565 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8190cdaf-43dd-4eae-9126-17e6bdc27221 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:33.508890 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8190cdaf-43dd-4eae-9126-17e6bdc27221 admin admin] PATH_INFO: `/v3/users` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:33.510831 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8190cdaf-43dd-4eae-9126-17e6bdc27221 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:58:33.517209 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8190cdaf-43dd-4eae-9126-17e6bdc27221 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:58:33.526346 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55c2ed1-fe90-49a2-b48a-cfef754b08b9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:58:33.530160 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55c2ed1-fe90-49a2-b48a-cfef754b08b9 service cinder] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:58:33.563153 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 857/1728] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 14:58:33 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:58:33.580337 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 872/1729] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Sep 1 14:58:33 2022] POST /identity/v3/users => generated 296 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:58:33.583518 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55c2ed1-fe90-49a2-b48a-cfef754b08b9 service cinder] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:58:33.603585 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55c2ed1-fe90-49a2-b48a-cfef754b08b9 service cinder] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:58:33.617438 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9891a3-052c-4245-b518-6820e5844d75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:58:33.618473 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9891a3-052c-4245-b518-6820e5844d75 admin admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:33.618912 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9891a3-052c-4245-b518-6820e5844d75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:33.619667 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9891a3-052c-4245-b518-6820e5844d75 admin admin] PATH_INFO: `/v3/roles` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:33.621312 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9891a3-052c-4245-b518-6820e5844d75 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:58:33.625127 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9891a3-052c-4245-b518-6820e5844d75 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:58:33.639869 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8190cdaf-43dd-4eae-9126-17e6bdc27221 admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:58:33.642378 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-8190cdaf-43dd-4eae-9126-17e6bdc27221 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:58:33.705343 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 858/1730] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 1 14:58:33 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 14:58:33.707420 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9891a3-052c-4245-b518-6820e5844d75 admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:58:33.745996 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9891a3-052c-4245-b518-6820e5844d75 admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:58:33.768358 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4e6003-765f-4771-b6ff-aa186460994b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:58:33.771485 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4e6003-765f-4771-b6ff-aa186460994b admin admin] REQUEST_METHOD: `PUT` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:33.775770 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4e6003-765f-4771-b6ff-aa186460994b admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:33.776612 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4e6003-765f-4771-b6ff-aa186460994b admin admin] PATH_INFO: `/v3/projects/c2511503309141808e6ed24da4338f4f/users/c2c53523ba1b463bbeebc3b54e283be2/roles/2a2adb769e55476b82e5014897e0a9c0` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:33.798983 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:58:33.841211 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4e6003-765f-4771-b6ff-aa186460994b admin admin] RBAC: Authorizing `identity:create_grant(project_id=c2511503309141808e6ed24da4338f4f, user_id=c2c53523ba1b463bbeebc3b54e283be2, role_id=2a2adb769e55476b82e5014897e0a9c0)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:58:33.857122 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4e6003-765f-4771-b6ff-aa186460994b admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:58:33.928391 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 859/1731] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 1 14:58:33 2022] PUT /identity/v3/projects/c2511503309141808e6ed24da4338f4f/users/c2c53523ba1b463bbeebc3b54e283be2/roles/2a2adb769e55476b82e5014897e0a9c0 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:58:33.933349 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4e6003-765f-4771-b6ff-aa186460994b admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:58:34.118672 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8190cdaf-43dd-4eae-9126-17e6bdc27221 admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:58:34.252051 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4e6003-765f-4771-b6ff-aa186460994b admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:58:34.294349 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5467214-3e33-4497-8676-ec9a8c049a54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:58:34.296162 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5467214-3e33-4497-8676-ec9a8c049a54 admin admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:34.297368 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5467214-3e33-4497-8676-ec9a8c049a54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:34.297711 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5467214-3e33-4497-8676-ec9a8c049a54 admin admin] PATH_INFO: `/v3/roles` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:34.301171 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5467214-3e33-4497-8676-ec9a8c049a54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:58:34.303896 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5467214-3e33-4497-8676-ec9a8c049a54 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:58:34.335172 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 860/1732] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 1 14:58:33 2022] GET /identity/v3/roles => generated 1420 bytes in 403 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:58:34.340284 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5467214-3e33-4497-8676-ec9a8c049a54 admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:58:34.344642 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b4cc5b-0aec-4d71-8a58-8e90efaf5087 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73a5bda5a6ea46e9863ccafd57dee339', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:58:34.346661 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b4cc5b-0aec-4d71-8a58-8e90efaf5087 service swift] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:34.347302 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b4cc5b-0aec-4d71-8a58-8e90efaf5087 service swift] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:34.350788 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b4cc5b-0aec-4d71-8a58-8e90efaf5087 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:34.375528 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5467214-3e33-4497-8676-ec9a8c049a54 admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:58:34.398033 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37549dca-d9fc-40ae-acff-d71de1f7c71b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:58:34.399735 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37549dca-d9fc-40ae-acff-d71de1f7c71b admin admin] REQUEST_METHOD: `PUT` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:34.401102 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37549dca-d9fc-40ae-acff-d71de1f7c71b admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:34.402906 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37549dca-d9fc-40ae-acff-d71de1f7c71b admin admin] PATH_INFO: `/v3/projects/c2511503309141808e6ed24da4338f4f/users/c2c53523ba1b463bbeebc3b54e283be2/roles/2a2adb769e55476b82e5014897e0a9c0` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:34.410675 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37549dca-d9fc-40ae-acff-d71de1f7c71b admin admin] RBAC: Authorizing `identity:create_grant(project_id=c2511503309141808e6ed24da4338f4f, user_id=c2c53523ba1b463bbeebc3b54e283be2, role_id=2a2adb769e55476b82e5014897e0a9c0)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:58:34.421099 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37549dca-d9fc-40ae-acff-d71de1f7c71b admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:58:34.470275 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 861/1733] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 1 14:58:34 2022] PUT /identity/v3/projects/c2511503309141808e6ed24da4338f4f/users/c2c53523ba1b463bbeebc3b54e283be2/roles/2a2adb769e55476b82e5014897e0a9c0 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:58:34.470275 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0231e02-17ae-45fc-a271-8117599ba59a None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:34.470275 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0231e02-17ae-45fc-a271-8117599ba59a None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:34.470275 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0231e02-17ae-45fc-a271-8117599ba59a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:34.542371 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-e0231e02-17ae-45fc-a271-8117599ba59a None None] MFA Rules not processed for user `c2c53523ba1b463bbeebc3b54e283be2`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:58:34.543227 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-e0231e02-17ae-45fc-a271-8117599ba59a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:58:34.546918 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b4cc5b-0aec-4d71-8a58-8e90efaf5087 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:58:34.552991 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b4cc5b-0aec-4d71-8a58-8e90efaf5087 service swift] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:58:34.589987 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 873/1734] 10.222.0.33 () {60 vars in 1384 bytes} [Thu Sep 1 14:58:33 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 951 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 01 14:58:34.784888 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:58:34.807149 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 862/1735] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 14:58:34 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 338 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:58:34.833873 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b4cc5b-0aec-4d71-8a58-8e90efaf5087 service swift] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:58:34.840939 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0231e02-17ae-45fc-a271-8117599ba59a None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:58:34.854695 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b4cc5b-0aec-4d71-8a58-8e90efaf5087 service swift] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:58:34.949480 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0231e02-17ae-45fc-a271-8117599ba59a None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:58:34.959001 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d24f5e-ca13-41b9-9025-8f9305ab7fd9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3b81739e34e4e648f88b6d73fcb5e46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:58:34.960256 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d24f5e-ca13-41b9-9025-8f9305ab7fd9 service cinder] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:34.961777 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d24f5e-ca13-41b9-9025-8f9305ab7fd9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:34.962304 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d24f5e-ca13-41b9-9025-8f9305ab7fd9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:34.995468 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d24f5e-ca13-41b9-9025-8f9305ab7fd9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:58:35.001266 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d24f5e-ca13-41b9-9025-8f9305ab7fd9 service cinder] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:58:35.045885 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 863/1736] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 14:58:34 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:58:35.075110 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:58:35.103930 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa478688-bc76-44cb-90aa-78caba96e797 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:58:35.114016 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa478688-bc76-44cb-90aa-78caba96e797 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:35.114016 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa478688-bc76-44cb-90aa-78caba96e797 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:35.114016 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa478688-bc76-44cb-90aa-78caba96e797 admin admin] PATH_INFO: `/v3/users/a38ba4fe45864fe880ea0f77a14f5591` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:35.114016 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa478688-bc76-44cb-90aa-78caba96e797 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a38ba4fe45864fe880ea0f77a14f5591)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:58:35.114567 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa478688-bc76-44cb-90aa-78caba96e797 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:58:35.236510 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-aa478688-bc76-44cb-90aa-78caba96e797 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a38ba4fe45864fe880ea0f77a14f5591'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:58:35.267762 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ee25a7-3bed-4058-b33e-3e6449e225ca None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:35.268254 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ee25a7-3bed-4058-b33e-3e6449e225ca None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:35.268475 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ee25a7-3bed-4058-b33e-3e6449e225ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:35.288876 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-aa478688-bc76-44cb-90aa-78caba96e797 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a38ba4fe45864fe880ea0f77a14f5591'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:58:35.291113 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-aa478688-bc76-44cb-90aa-78caba96e797 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a38ba4fe45864fe880ea0f77a14f5591'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:58:35.352690 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-aa478688-bc76-44cb-90aa-78caba96e797 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a38ba4fe45864fe880ea0f77a14f5591'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:58:35.355177 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-aa478688-bc76-44cb-90aa-78caba96e797 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a38ba4fe45864fe880ea0f77a14f5591'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:58:35.371385 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-76ee25a7-3bed-4058-b33e-3e6449e225ca None None] MFA Rules not processed for user `229e076684dd4bbdacda2231b18f0875`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:58:35.375318 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-76ee25a7-3bed-4058-b33e-3e6449e225ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:58:35.415373 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 874/1737] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Sep 1 14:58:34 2022] DELETE /identity/v3/users/a38ba4fe45864fe880ea0f77a14f5591 => generated 0 bytes in 580 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:58:35.417163 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ec15a5-2f01-408e-b1a6-826e9309bacc None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:35.417671 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ec15a5-2f01-408e-b1a6-826e9309bacc None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:35.420388 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ec15a5-2f01-408e-b1a6-826e9309bacc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:35.496790 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-62ec15a5-2f01-408e-b1a6-826e9309bacc None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:58:35.501727 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-62ec15a5-2f01-408e-b1a6-826e9309bacc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:58:35.531239 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 864/1738] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 14:58:35 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 14:58:35.550951 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ee25a7-3bed-4058-b33e-3e6449e225ca None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:58:35.553390 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-76ee25a7-3bed-4058-b33e-3e6449e225ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:58:35.667086 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 875/1739] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 14:58:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:58:35.705308 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ec15a5-2f01-408e-b1a6-826e9309bacc None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:58:35.706552 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-62ec15a5-2f01-408e-b1a6-826e9309bacc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:58:35.802871 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ee25a7-3bed-4058-b33e-3e6449e225ca None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:58:35.854683 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ec15a5-2f01-408e-b1a6-826e9309bacc None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:58:35.861879 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:58:35.866939 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648bffc9-93f2-46bc-8406-a1caeea94b9f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2790149639aa418bbc4e4982e93eda44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:58:35.867654 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648bffc9-93f2-46bc-8406-a1caeea94b9f service neutron] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:35.868001 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648bffc9-93f2-46bc-8406-a1caeea94b9f service neutron] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:35.868361 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648bffc9-93f2-46bc-8406-a1caeea94b9f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:35.882619 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648bffc9-93f2-46bc-8406-a1caeea94b9f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:58:35.884971 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648bffc9-93f2-46bc-8406-a1caeea94b9f service neutron] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:58:35.901018 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 876/1740] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 14:58:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 14:58:35.947054 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d425299-a6d6-4c1e-a0d9-384fdc512de7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4858bb8621854ec8aba836e7d9050498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:58:35.949493 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d425299-a6d6-4c1e-a0d9-384fdc512de7 service nova] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:35.949778 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d425299-a6d6-4c1e-a0d9-384fdc512de7 service nova] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:35.950055 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d425299-a6d6-4c1e-a0d9-384fdc512de7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:36.091223 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d425299-a6d6-4c1e-a0d9-384fdc512de7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:58:36.094198 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d425299-a6d6-4c1e-a0d9-384fdc512de7 service nova] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:58:36.126290 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 865/1741] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 14:58:35 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 576 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 14:58:36.167070 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648bffc9-93f2-46bc-8406-a1caeea94b9f service neutron] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:58:36.168547 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-648bffc9-93f2-46bc-8406-a1caeea94b9f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:58:36.190152 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648bffc9-93f2-46bc-8406-a1caeea94b9f service neutron] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:58:36.216262 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb35daa1-89c5-40b5-858b-eb444544481b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:58:36.216915 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb35daa1-89c5-40b5-858b-eb444544481b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:36.217199 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb35daa1-89c5-40b5-858b-eb444544481b admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:36.217649 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb35daa1-89c5-40b5-858b-eb444544481b admin admin] PATH_INFO: `/v3/users/b8d124cf913b46babd8a9d0a528ee3b0` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:36.224952 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb35daa1-89c5-40b5-858b-eb444544481b admin admin] RBAC: Authorizing `identity:delete_user(user_id=b8d124cf913b46babd8a9d0a528ee3b0)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:58:36.226984 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb35daa1-89c5-40b5-858b-eb444544481b admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:58:36.273482 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:58:36.386969 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-eb35daa1-89c5-40b5-858b-eb444544481b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b8d124cf913b46babd8a9d0a528ee3b0'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:58:36.415508 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-eb35daa1-89c5-40b5-858b-eb444544481b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b8d124cf913b46babd8a9d0a528ee3b0'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:58:36.417043 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-eb35daa1-89c5-40b5-858b-eb444544481b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b8d124cf913b46babd8a9d0a528ee3b0'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:58:36.464570 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-eb35daa1-89c5-40b5-858b-eb444544481b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b8d124cf913b46babd8a9d0a528ee3b0'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:58:36.469050 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-eb35daa1-89c5-40b5-858b-eb444544481b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b8d124cf913b46babd8a9d0a528ee3b0'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:58:36.492080 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 877/1742] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Sep 1 14:58:36 2022] DELETE /identity/v3/users/b8d124cf913b46babd8a9d0a528ee3b0 => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:58:36.496586 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57464bca-2dbc-49cc-96d4-4a9f8818f5d0 None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:36.497136 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57464bca-2dbc-49cc-96d4-4a9f8818f5d0 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:36.497136 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57464bca-2dbc-49cc-96d4-4a9f8818f5d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:36.556517 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-57464bca-2dbc-49cc-96d4-4a9f8818f5d0 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:58:36.559319 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-57464bca-2dbc-49cc-96d4-4a9f8818f5d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:58:36.746148 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 866/1743] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 14:58:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:58:36.850612 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb35daa1-89c5-40b5-858b-eb444544481b admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:58:36.852189 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-eb35daa1-89c5-40b5-858b-eb444544481b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:58:37.029739 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb35daa1-89c5-40b5-858b-eb444544481b admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:58:37.037112 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa7be33-40fe-4c57-9e3e-d03d399f4a21 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2790149639aa418bbc4e4982e93eda44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:58:37.037763 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa7be33-40fe-4c57-9e3e-d03d399f4a21 service neutron] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:37.038039 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa7be33-40fe-4c57-9e3e-d03d399f4a21 service neutron] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:37.038401 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa7be33-40fe-4c57-9e3e-d03d399f4a21 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:37.054582 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa7be33-40fe-4c57-9e3e-d03d399f4a21 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:58:37.056676 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa7be33-40fe-4c57-9e3e-d03d399f4a21 service neutron] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:58:37.076265 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 878/1744] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 14:58:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:58:37.341023 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6b28de-76b8-4ada-ae5c-2dc8277dbbf1 None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:37.341566 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6b28de-76b8-4ada-ae5c-2dc8277dbbf1 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:37.342039 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6b28de-76b8-4ada-ae5c-2dc8277dbbf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:37.436025 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-8e6b28de-76b8-4ada-ae5c-2dc8277dbbf1 None None] MFA Rules not processed for user `df75151c37f74b6da7428dc290a274ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:58:37.439571 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-8e6b28de-76b8-4ada-ae5c-2dc8277dbbf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:58:37.761490 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:58:37.795713 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 867/1745] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 14:58:37 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 456 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:58:37.808119 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30dcc03-1922-4b2b-a543-71c2593b4456 None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:37.808691 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30dcc03-1922-4b2b-a543-71c2593b4456 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:37.808691 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30dcc03-1922-4b2b-a543-71c2593b4456 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:37.859884 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-c30dcc03-1922-4b2b-a543-71c2593b4456 None None] MFA Rules not processed for user `df75151c37f74b6da7428dc290a274ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:58:37.865529 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-c30dcc03-1922-4b2b-a543-71c2593b4456 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:58:37.928285 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 879/1746] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 14:58:37 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:58:37.943980 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6b28de-76b8-4ada-ae5c-2dc8277dbbf1 None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:58:37.945908 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-8e6b28de-76b8-4ada-ae5c-2dc8277dbbf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:58:38.155641 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6b28de-76b8-4ada-ae5c-2dc8277dbbf1 None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:58:38.168381 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e5f19e-97c2-415f-b737-b8819da46792 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4858bb8621854ec8aba836e7d9050498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:58:38.169794 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e5f19e-97c2-415f-b737-b8819da46792 service nova] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:38.170154 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e5f19e-97c2-415f-b737-b8819da46792 service nova] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:38.170379 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e5f19e-97c2-415f-b737-b8819da46792 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:38.200493 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e5f19e-97c2-415f-b737-b8819da46792 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:58:38.204041 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e5f19e-97c2-415f-b737-b8819da46792 service nova] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:58:38.234465 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 868/1747] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 14:58:37 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:58:38.824260 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d83db2a-ff05-487e-ac90-91c79967b36d None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:38.824891 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d83db2a-ff05-487e-ac90-91c79967b36d None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:38.825696 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d83db2a-ff05-487e-ac90-91c79967b36d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:38.913351 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-4d83db2a-ff05-487e-ac90-91c79967b36d None None] MFA Rules not processed for user `82255a5060674d77a554e34f89ab1bff`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:58:38.916091 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-4d83db2a-ff05-487e-ac90-91c79967b36d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:58:39.052989 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e5f19e-97c2-415f-b737-b8819da46792 service nova] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:58:39.055618 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-06e5f19e-97c2-415f-b737-b8819da46792 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:58:39.094266 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e5f19e-97c2-415f-b737-b8819da46792 service nova] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:58:39.123282 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55637c5-ac6b-4061-af23-235e43e6b9d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:58:39.124661 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55637c5-ac6b-4061-af23-235e43e6b9d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:39.125178 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55637c5-ac6b-4061-af23-235e43e6b9d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:39.125655 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55637c5-ac6b-4061-af23-235e43e6b9d1 admin admin] PATH_INFO: `/v3/projects/a59b1cce594f41f8a7e1f4f36e8f3386` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:39.128651 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55637c5-ac6b-4061-af23-235e43e6b9d1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a59b1cce594f41f8a7e1f4f36e8f3386)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:58:39.138819 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55637c5-ac6b-4061-af23-235e43e6b9d1 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:58:39.144972 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 880/1748] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 14:58:38 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 325 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:58:39.164262 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d83db2a-ff05-487e-ac90-91c79967b36d None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:58:39.167262 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-4d83db2a-ff05-487e-ac90-91c79967b36d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:58:39.219315 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d83db2a-ff05-487e-ac90-91c79967b36d None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:58:39.230467 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659cdd95-f8dd-4ebd-8d1a-cfaa1b52e6cb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4858bb8621854ec8aba836e7d9050498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:58:39.232173 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659cdd95-f8dd-4ebd-8d1a-cfaa1b52e6cb service nova] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:39.232455 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659cdd95-f8dd-4ebd-8d1a-cfaa1b52e6cb service nova] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:39.235744 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659cdd95-f8dd-4ebd-8d1a-cfaa1b52e6cb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:39.249511 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-b55637c5-ac6b-4061-af23-235e43e6b9d1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a59b1cce594f41f8a7e1f4f36e8f3386', 'request_id': 'req-b55637c5-ac6b-4061-af23-235e43e6b9d1'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:58:39.258193 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-b55637c5-ac6b-4061-af23-235e43e6b9d1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a59b1cce594f41f8a7e1f4f36e8f3386', 'request_id': 'req-b55637c5-ac6b-4061-af23-235e43e6b9d1'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:58:39.261537 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659cdd95-f8dd-4ebd-8d1a-cfaa1b52e6cb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:58:39.265725 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659cdd95-f8dd-4ebd-8d1a-cfaa1b52e6cb service nova] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:58:39.269919 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-b55637c5-ac6b-4061-af23-235e43e6b9d1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a59b1cce594f41f8a7e1f4f36e8f3386', 'request_id': 'req-b55637c5-ac6b-4061-af23-235e43e6b9d1'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:58:39.281666 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:58:39.289256 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-b55637c5-ac6b-4061-af23-235e43e6b9d1 admin admin] The token cache is being invalidate because project a59b1cce594f41f8a7e1f4f36e8f3386 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81341) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:58:39.289609 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-b55637c5-ac6b-4061-af23-235e43e6b9d1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:58:39.290592 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-b55637c5-ac6b-4061-af23-235e43e6b9d1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:58:39.297348 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 869/1749] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Sep 1 14:58:39 2022] DELETE /identity/v3/projects/a59b1cce594f41f8a7e1f4f36e8f3386 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:58:39.299247 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55637c5-ac6b-4061-af23-235e43e6b9d1 admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:58:39.301427 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-b55637c5-ac6b-4061-af23-235e43e6b9d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:58:39.322813 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 881/1750] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 14:58:39 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:58:39.362392 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659cdd95-f8dd-4ebd-8d1a-cfaa1b52e6cb service nova] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:58:39.365766 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-659cdd95-f8dd-4ebd-8d1a-cfaa1b52e6cb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:58:39.389663 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659cdd95-f8dd-4ebd-8d1a-cfaa1b52e6cb service nova] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:58:39.425967 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55637c5-ac6b-4061-af23-235e43e6b9d1 admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:58:39.440324 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbd5e91-d326-471f-ab0a-1e7c7af015e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:58:39.441095 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbd5e91-d326-471f-ab0a-1e7c7af015e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:39.441532 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbd5e91-d326-471f-ab0a-1e7c7af015e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:39.441866 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbd5e91-d326-471f-ab0a-1e7c7af015e0 admin admin] PATH_INFO: `/v3/users/c2c53523ba1b463bbeebc3b54e283be2` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:39.447004 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbd5e91-d326-471f-ab0a-1e7c7af015e0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c2c53523ba1b463bbeebc3b54e283be2)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:58:39.451927 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbd5e91-d326-471f-ab0a-1e7c7af015e0 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:58:39.467836 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:58:39.562370 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d30d9e-d4bf-4d5a-846f-f310f4071f50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:58:39.563032 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d30d9e-d4bf-4d5a-846f-f310f4071f50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:39.563317 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d30d9e-d4bf-4d5a-846f-f310f4071f50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:39.563596 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d30d9e-d4bf-4d5a-846f-f310f4071f50 admin admin] PATH_INFO: `/v3/projects/055f989992444be394ef45c6fdbb6a53` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:39.569176 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d30d9e-d4bf-4d5a-846f-f310f4071f50 admin admin] RBAC: Authorizing `identity:delete_project(project_id=055f989992444be394ef45c6fdbb6a53)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:58:39.571872 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d30d9e-d4bf-4d5a-846f-f310f4071f50 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:58:39.574508 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-efbd5e91-d326-471f-ab0a-1e7c7af015e0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c2c53523ba1b463bbeebc3b54e283be2'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:58:39.593414 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-efbd5e91-d326-471f-ab0a-1e7c7af015e0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c2c53523ba1b463bbeebc3b54e283be2'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:58:39.594256 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-efbd5e91-d326-471f-ab0a-1e7c7af015e0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c2c53523ba1b463bbeebc3b54e283be2'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:58:39.635969 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-efbd5e91-d326-471f-ab0a-1e7c7af015e0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c2c53523ba1b463bbeebc3b54e283be2'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:58:39.638226 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-efbd5e91-d326-471f-ab0a-1e7c7af015e0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c2c53523ba1b463bbeebc3b54e283be2'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:58:39.681754 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 870/1751] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Sep 1 14:58:39 2022] DELETE /identity/v3/users/c2c53523ba1b463bbeebc3b54e283be2 => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:58:39.686398 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7a1906-82bb-4c1f-ae6c-c97f4a16088e None None] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:39.699844 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7a1906-82bb-4c1f-ae6c-c97f4a16088e None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:39.700213 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7a1906-82bb-4c1f-ae6c-c97f4a16088e None None] PATH_INFO: `/` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:39.704819 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 871/1752] 10.222.0.33 () {54 vars in 912 bytes} [Thu Sep 1 14:58:39 2022] GET /identity => generated 270 bytes in 21 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:58:39.717146 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08cec3d-83f6-4dcb-b9cd-6c7a57ae2ff0 None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:39.717435 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08cec3d-83f6-4dcb-b9cd-6c7a57ae2ff0 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:39.717713 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08cec3d-83f6-4dcb-b9cd-6c7a57ae2ff0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:39.744131 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-07d30d9e-d4bf-4d5a-846f-f310f4071f50 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '055f989992444be394ef45c6fdbb6a53', 'request_id': 'req-07d30d9e-d4bf-4d5a-846f-f310f4071f50'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:58:39.760354 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-07d30d9e-d4bf-4d5a-846f-f310f4071f50 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '055f989992444be394ef45c6fdbb6a53', 'request_id': 'req-07d30d9e-d4bf-4d5a-846f-f310f4071f50'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:58:39.786678 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-07d30d9e-d4bf-4d5a-846f-f310f4071f50 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '055f989992444be394ef45c6fdbb6a53', 'request_id': 'req-07d30d9e-d4bf-4d5a-846f-f310f4071f50'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:58:39.826259 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-07d30d9e-d4bf-4d5a-846f-f310f4071f50 admin admin] The token cache is being invalidate because project 055f989992444be394ef45c6fdbb6a53 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81342) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:58:39.826898 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-07d30d9e-d4bf-4d5a-846f-f310f4071f50 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:58:39.827771 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-07d30d9e-d4bf-4d5a-846f-f310f4071f50 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:58:39.837475 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 882/1753] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Sep 1 14:58:39 2022] DELETE /identity/v3/projects/055f989992444be394ef45c6fdbb6a53 => generated 0 bytes in 475 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:58:39.841294 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d30d9e-d4bf-4d5a-846f-f310f4071f50 admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:58:39.844134 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-07d30d9e-d4bf-4d5a-846f-f310f4071f50 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:58:39.889291 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-c08cec3d-83f6-4dcb-b9cd-6c7a57ae2ff0 None None] MFA Rules not processed for user `2790149639aa418bbc4e4982e93eda44`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:58:39.898085 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-c08cec3d-83f6-4dcb-b9cd-6c7a57ae2ff0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:58:39.991994 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d30d9e-d4bf-4d5a-846f-f310f4071f50 admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:58:39.997975 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8debb68-4c12-4236-9cfe-abc1b373f55a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:58:39.998908 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8debb68-4c12-4236-9cfe-abc1b373f55a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:40.000385 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8debb68-4c12-4236-9cfe-abc1b373f55a admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:40.001056 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8debb68-4c12-4236-9cfe-abc1b373f55a admin admin] PATH_INFO: `/v3/users/d96325e5eace4e8799cd40dcfd8b3cb0` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:40.008518 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8debb68-4c12-4236-9cfe-abc1b373f55a admin admin] RBAC: Authorizing `identity:delete_user(user_id=d96325e5eace4e8799cd40dcfd8b3cb0)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:58:40.010843 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8debb68-4c12-4236-9cfe-abc1b373f55a admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:58:40.098482 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 872/1754] 10.222.0.33 () {58 vars in 1009 bytes} [Thu Sep 1 14:58:39 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 385 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:58:40.110901 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21357a1f-f6df-4914-bb55-8677db458e5b None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:40.111453 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21357a1f-f6df-4914-bb55-8677db458e5b None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:40.113185 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21357a1f-f6df-4914-bb55-8677db458e5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:40.114719 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:58:40.169866 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-b8debb68-4c12-4236-9cfe-abc1b373f55a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd96325e5eace4e8799cd40dcfd8b3cb0'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:58:40.188373 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-b8debb68-4c12-4236-9cfe-abc1b373f55a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd96325e5eace4e8799cd40dcfd8b3cb0'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:58:40.189603 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-b8debb68-4c12-4236-9cfe-abc1b373f55a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd96325e5eace4e8799cd40dcfd8b3cb0'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:58:40.215983 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-b8debb68-4c12-4236-9cfe-abc1b373f55a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd96325e5eace4e8799cd40dcfd8b3cb0'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:58:40.216781 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-b8debb68-4c12-4236-9cfe-abc1b373f55a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd96325e5eace4e8799cd40dcfd8b3cb0'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:58:40.223343 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-21357a1f-f6df-4914-bb55-8677db458e5b None None] MFA Rules not processed for user `bb7098b844d743f5bafeb7ba15b3fdd7`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:58:40.224625 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-21357a1f-f6df-4914-bb55-8677db458e5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:58:40.258779 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 883/1755] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Sep 1 14:58:39 2022] DELETE /identity/v3/users/d96325e5eace4e8799cd40dcfd8b3cb0 => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:58:40.352865 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8debb68-4c12-4236-9cfe-abc1b373f55a admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:58:40.356758 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-b8debb68-4c12-4236-9cfe-abc1b373f55a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:58:40.388755 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 873/1756] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 14:58:40 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 289 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:58:40.400570 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cc0b0b-8184-42ee-aedc-dab6f02165ec None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:40.400963 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cc0b0b-8184-42ee-aedc-dab6f02165ec None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:40.401343 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cc0b0b-8184-42ee-aedc-dab6f02165ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:40.447192 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-e8cc0b0b-8184-42ee-aedc-dab6f02165ec None None] MFA Rules not processed for user `bb7098b844d743f5bafeb7ba15b3fdd7`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:58:40.458646 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-e8cc0b0b-8184-42ee-aedc-dab6f02165ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:58:40.469404 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:58:40.540744 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8debb68-4c12-4236-9cfe-abc1b373f55a admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:58:40.544817 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c5d992-2be5-4df4-bba0-f0c23a1ef0d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2790149639aa418bbc4e4982e93eda44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:58:40.545448 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c5d992-2be5-4df4-bba0-f0c23a1ef0d5 service neutron] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:40.545729 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c5d992-2be5-4df4-bba0-f0c23a1ef0d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:40.546004 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c5d992-2be5-4df4-bba0-f0c23a1ef0d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:40.552419 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-a6c5d992-2be5-4df4-bba0-f0c23a1ef0d5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:58:40.573637 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c5d992-2be5-4df4-bba0-f0c23a1ef0d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:58:40.578969 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c5d992-2be5-4df4-bba0-f0c23a1ef0d5 service neutron] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:58:40.605851 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 884/1757] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 14:58:40 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 253 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:58:40.622279 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 874/1758] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 14:58:40 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:58:40.631081 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c5d992-2be5-4df4-bba0-f0c23a1ef0d5 service neutron] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:58:40.635876 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-a6c5d992-2be5-4df4-bba0-f0c23a1ef0d5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:58:40.752203 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c5d992-2be5-4df4-bba0-f0c23a1ef0d5 service neutron] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:58:40.766721 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe089cf7-8a5f-4e86-8e63-0caefb60c2d8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4858bb8621854ec8aba836e7d9050498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:58:40.768226 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe089cf7-8a5f-4e86-8e63-0caefb60c2d8 service nova] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:40.768509 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe089cf7-8a5f-4e86-8e63-0caefb60c2d8 service nova] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:40.770933 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe089cf7-8a5f-4e86-8e63-0caefb60c2d8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:40.812755 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe089cf7-8a5f-4e86-8e63-0caefb60c2d8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:58:40.816800 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe089cf7-8a5f-4e86-8e63-0caefb60c2d8 service nova] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:58:40.839439 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 885/1759] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 14:58:40 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:58:40.952159 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cc0b0b-8184-42ee-aedc-dab6f02165ec None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:58:40.954014 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-e8cc0b0b-8184-42ee-aedc-dab6f02165ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:58:41.140051 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cc0b0b-8184-42ee-aedc-dab6f02165ec None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:58:41.150517 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60133ef-9182-4b54-ba7f-b95ec2ef01a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:58:41.151178 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60133ef-9182-4b54-ba7f-b95ec2ef01a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:41.151517 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60133ef-9182-4b54-ba7f-b95ec2ef01a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:41.151784 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60133ef-9182-4b54-ba7f-b95ec2ef01a1 admin admin] PATH_INFO: `/v3/projects/c2511503309141808e6ed24da4338f4f` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:41.154288 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60133ef-9182-4b54-ba7f-b95ec2ef01a1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c2511503309141808e6ed24da4338f4f)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:58:41.169811 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60133ef-9182-4b54-ba7f-b95ec2ef01a1 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:58:41.292515 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-b60133ef-9182-4b54-ba7f-b95ec2ef01a1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c2511503309141808e6ed24da4338f4f', 'request_id': 'req-b60133ef-9182-4b54-ba7f-b95ec2ef01a1'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:58:41.312850 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-b60133ef-9182-4b54-ba7f-b95ec2ef01a1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c2511503309141808e6ed24da4338f4f', 'request_id': 'req-b60133ef-9182-4b54-ba7f-b95ec2ef01a1'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:58:41.343411 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-b60133ef-9182-4b54-ba7f-b95ec2ef01a1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c2511503309141808e6ed24da4338f4f', 'request_id': 'req-b60133ef-9182-4b54-ba7f-b95ec2ef01a1'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:58:41.416423 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-b60133ef-9182-4b54-ba7f-b95ec2ef01a1 admin admin] The token cache is being invalidate because project c2511503309141808e6ed24da4338f4f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81341) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:58:41.418162 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-b60133ef-9182-4b54-ba7f-b95ec2ef01a1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:58:41.419403 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-b60133ef-9182-4b54-ba7f-b95ec2ef01a1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:58:41.422137 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 875/1760] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Sep 1 14:58:40 2022] DELETE /identity/v3/projects/c2511503309141808e6ed24da4338f4f => generated 0 bytes in 471 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:58:41.427901 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb65dc99-92ae-4cfd-80b9-cef3fc2e485f None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:41.428183 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb65dc99-92ae-4cfd-80b9-cef3fc2e485f None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:41.428826 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb65dc99-92ae-4cfd-80b9-cef3fc2e485f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:41.508635 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-cb65dc99-92ae-4cfd-80b9-cef3fc2e485f None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:58:41.517168 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-cb65dc99-92ae-4cfd-80b9-cef3fc2e485f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:58:41.604195 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:58:41.677706 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 886/1761] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 14:58:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:58:41.688346 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517c993e-a693-4016-b333-7239c6066a6c None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:41.688346 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517c993e-a693-4016-b333-7239c6066a6c None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:41.688346 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517c993e-a693-4016-b333-7239c6066a6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:41.697079 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb65dc99-92ae-4cfd-80b9-cef3fc2e485f None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:58:41.700588 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-cb65dc99-92ae-4cfd-80b9-cef3fc2e485f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:58:41.738362 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb65dc99-92ae-4cfd-80b9-cef3fc2e485f None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:58:41.740713 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-517c993e-a693-4016-b333-7239c6066a6c None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:58:41.744861 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-517c993e-a693-4016-b333-7239c6066a6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:58:41.755872 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b239758-f63d-4e5a-b508-1ea351e87cf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:58:41.756841 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b239758-f63d-4e5a-b508-1ea351e87cf8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:41.757469 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b239758-f63d-4e5a-b508-1ea351e87cf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:41.757979 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b239758-f63d-4e5a-b508-1ea351e87cf8 admin admin] PATH_INFO: `/v3/projects/298db1d1c9164f93af26f4ded1819ce1` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:41.763427 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 876/1762] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 14:58:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:58:41.764750 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517c993e-a693-4016-b333-7239c6066a6c None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:58:41.765060 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b239758-f63d-4e5a-b508-1ea351e87cf8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=298db1d1c9164f93af26f4ded1819ce1)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:58:41.770620 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b239758-f63d-4e5a-b508-1ea351e87cf8 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:58:41.794836 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517c993e-a693-4016-b333-7239c6066a6c None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:58:41.810659 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8172d8b-c145-48f3-ad4d-097bfde9eabf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:58:41.811309 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8172d8b-c145-48f3-ad4d-097bfde9eabf admin admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:41.813826 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8172d8b-c145-48f3-ad4d-097bfde9eabf admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:41.814117 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8172d8b-c145-48f3-ad4d-097bfde9eabf admin admin] PATH_INFO: `/v3/domains` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:41.814889 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8172d8b-c145-48f3-ad4d-097bfde9eabf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81341) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:58:41.815124 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8172d8b-c145-48f3-ad4d-097bfde9eabf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:58:41.821439 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8172d8b-c145-48f3-ad4d-097bfde9eabf admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:58:41.862984 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 877/1763] 10.222.0.33 () {58 vars in 1153 bytes} [Thu Sep 1 14:58:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:58:41.867563 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8172d8b-c145-48f3-ad4d-097bfde9eabf admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:58:41.895541 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8172d8b-c145-48f3-ad4d-097bfde9eabf admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:58:41.922902 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-9b239758-f63d-4e5a-b508-1ea351e87cf8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '298db1d1c9164f93af26f4ded1819ce1', 'request_id': 'req-9b239758-f63d-4e5a-b508-1ea351e87cf8'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:58:41.983699 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-9b239758-f63d-4e5a-b508-1ea351e87cf8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '298db1d1c9164f93af26f4ded1819ce1', 'request_id': 'req-9b239758-f63d-4e5a-b508-1ea351e87cf8'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:58:42.005947 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:58:42.029548 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-9b239758-f63d-4e5a-b508-1ea351e87cf8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '298db1d1c9164f93af26f4ded1819ce1', 'request_id': 'req-9b239758-f63d-4e5a-b508-1ea351e87cf8'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:58:42.083731 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-9b239758-f63d-4e5a-b508-1ea351e87cf8 admin admin] The token cache is being invalidate because project 298db1d1c9164f93af26f4ded1819ce1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81342) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:58:42.089338 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-9b239758-f63d-4e5a-b508-1ea351e87cf8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:58:42.091390 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-9b239758-f63d-4e5a-b508-1ea351e87cf8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:58:42.104088 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 887/1764] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Sep 1 14:58:41 2022] DELETE /identity/v3/projects/298db1d1c9164f93af26f4ded1819ce1 => generated 0 bytes in 404 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:58:42.105055 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b239758-f63d-4e5a-b508-1ea351e87cf8 admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:58:42.111392 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-9b239758-f63d-4e5a-b508-1ea351e87cf8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:58:42.133977 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc9e3d7-e45a-49a0-939f-100dd865ec48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:58:42.134656 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc9e3d7-e45a-49a0-939f-100dd865ec48 admin admin] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:42.134929 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc9e3d7-e45a-49a0-939f-100dd865ec48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:42.137261 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc9e3d7-e45a-49a0-939f-100dd865ec48 admin admin] PATH_INFO: `/v3/projects` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:42.139092 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc9e3d7-e45a-49a0-939f-100dd865ec48 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:58:42.149989 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc9e3d7-e45a-49a0-939f-100dd865ec48 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:58:42.209603 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 878/1765] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Sep 1 14:58:41 2022] POST /identity/v3/projects => generated 381 bytes in 344 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:58:42.210987 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7be2b28-3439-429a-a6f0-c0cfbf44865f None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:42.214019 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7be2b28-3439-429a-a6f0-c0cfbf44865f None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:42.214364 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7be2b28-3439-429a-a6f0-c0cfbf44865f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:42.290307 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-c7be2b28-3439-429a-a6f0-c0cfbf44865f None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:58:42.294023 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-c7be2b28-3439-429a-a6f0-c0cfbf44865f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:58:42.367710 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b239758-f63d-4e5a-b508-1ea351e87cf8 admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:58:42.498218 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 879/1766] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 14:58:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 288 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:58:42.501583 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae8ee8a-332c-495b-8326-e69952f703e5 None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:42.505285 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae8ee8a-332c-495b-8326-e69952f703e5 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:42.505571 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae8ee8a-332c-495b-8326-e69952f703e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:42.543283 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d8cd45-95d5-4c75-bf62-7a9c774e315e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f312c8dcabe48ef9ea453364ade38ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:58:42.544099 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d8cd45-95d5-4c75-bf62-7a9c774e315e service placement] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:42.544381 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d8cd45-95d5-4c75-bf62-7a9c774e315e service placement] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:42.562781 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d8cd45-95d5-4c75-bf62-7a9c774e315e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:42.570593 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-94d8cd45-95d5-4c75-bf62-7a9c774e315e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:58:42.588977 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:58:42.595384 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d8cd45-95d5-4c75-bf62-7a9c774e315e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:58:42.597205 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d8cd45-95d5-4c75-bf62-7a9c774e315e service placement] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:58:42.600198 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-cae8ee8a-332c-495b-8326-e69952f703e5 None None] MFA Rules not processed for user `df75151c37f74b6da7428dc290a274ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:58:42.603272 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-cae8ee8a-332c-495b-8326-e69952f703e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:58:42.615952 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 888/1767] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 14:58:42 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 511 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:58:42.616946 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d8cd45-95d5-4c75-bf62-7a9c774e315e service placement] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:58:42.618794 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-94d8cd45-95d5-4c75-bf62-7a9c774e315e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:58:42.636821 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d8cd45-95d5-4c75-bf62-7a9c774e315e service placement] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:58:42.642429 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe57811a-fea8-45e3-b587-d60e9b40b684 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:58:42.643253 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe57811a-fea8-45e3-b587-d60e9b40b684 admin admin] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:42.643640 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe57811a-fea8-45e3-b587-d60e9b40b684 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:42.644544 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe57811a-fea8-45e3-b587-d60e9b40b684 admin admin] PATH_INFO: `/v3/users` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:42.644742 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe57811a-fea8-45e3-b587-d60e9b40b684 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:58:42.647033 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe57811a-fea8-45e3-b587-d60e9b40b684 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:58:42.691733 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 889/1768] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Sep 1 14:58:42 2022] POST /identity/v3/users => generated 291 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:58:42.696890 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd65cbd1-5a20-49be-b2c8-d59d2a793f0b None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:42.698695 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd65cbd1-5a20-49be-b2c8-d59d2a793f0b None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:42.699353 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd65cbd1-5a20-49be-b2c8-d59d2a793f0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:42.728949 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-fd65cbd1-5a20-49be-b2c8-d59d2a793f0b None None] MFA Rules not processed for user `229e076684dd4bbdacda2231b18f0875`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:58:42.730187 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-fd65cbd1-5a20-49be-b2c8-d59d2a793f0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:58:42.785025 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 880/1769] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 14:58:42 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 286 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:58:42.790605 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42027cf-9b09-4dd0-ae2c-2f87bb58bfad None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:42.790884 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42027cf-9b09-4dd0-ae2c-2f87bb58bfad None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:42.791163 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42027cf-9b09-4dd0-ae2c-2f87bb58bfad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:42.914510 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-b42027cf-9b09-4dd0-ae2c-2f87bb58bfad None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:58:42.915762 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-b42027cf-9b09-4dd0-ae2c-2f87bb58bfad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:58:42.951590 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 881/1770] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 14:58:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:58:42.952340 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42027cf-9b09-4dd0-ae2c-2f87bb58bfad None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:58:42.976326 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42027cf-9b09-4dd0-ae2c-2f87bb58bfad None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:58:42.992129 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570abdc0-cf00-4dfa-a23b-80b5d661411e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:58:42.992129 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570abdc0-cf00-4dfa-a23b-80b5d661411e admin admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:42.992304 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570abdc0-cf00-4dfa-a23b-80b5d661411e admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:42.992532 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570abdc0-cf00-4dfa-a23b-80b5d661411e admin admin] PATH_INFO: `/v3/roles` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:42.993185 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570abdc0-cf00-4dfa-a23b-80b5d661411e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:58:43.006904 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570abdc0-cf00-4dfa-a23b-80b5d661411e admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:58:43.030885 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 882/1771] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 1 14:58:42 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:58:43.032592 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1968c5f7-77fc-435f-b8e6-c2cfa53c83b2 None None] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:43.034347 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1968c5f7-77fc-435f-b8e6-c2cfa53c83b2 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:43.034893 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1968c5f7-77fc-435f-b8e6-c2cfa53c83b2 None None] PATH_INFO: `/` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:43.045457 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 883/1772] 10.222.0.33 () {54 vars in 912 bytes} [Thu Sep 1 14:58:43 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:58:43.048175 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 890/1773] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 14:58:42 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 356 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:58:43.050447 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1968c5f7-77fc-435f-b8e6-c2cfa53c83b2 None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:58:43.052188 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd65cbd1-5a20-49be-b2c8-d59d2a793f0b None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:58:43.081714 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1968c5f7-77fc-435f-b8e6-c2cfa53c83b2 None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:58:43.093239 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd65cbd1-5a20-49be-b2c8-d59d2a793f0b None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:58:43.093374 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71694858-5f35-4d1c-9783-01ae1e1abe07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:58:43.093971 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71694858-5f35-4d1c-9783-01ae1e1abe07 admin admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:43.094319 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71694858-5f35-4d1c-9783-01ae1e1abe07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:43.094707 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71694858-5f35-4d1c-9783-01ae1e1abe07 admin admin] PATH_INFO: `/v3/domains` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:43.095360 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71694858-5f35-4d1c-9783-01ae1e1abe07 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81341) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:58:43.095658 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71694858-5f35-4d1c-9783-01ae1e1abe07 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:58:43.097280 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71694858-5f35-4d1c-9783-01ae1e1abe07 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:58:43.114970 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7876cc8d-7b05-4b36-92f2-7e7f5b64dbca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:58:43.115617 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7876cc8d-7b05-4b36-92f2-7e7f5b64dbca admin admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:43.115906 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7876cc8d-7b05-4b36-92f2-7e7f5b64dbca admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:43.116177 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7876cc8d-7b05-4b36-92f2-7e7f5b64dbca admin admin] PATH_INFO: `/v3/roles` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:43.116851 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7876cc8d-7b05-4b36-92f2-7e7f5b64dbca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:58:43.120315 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7876cc8d-7b05-4b36-92f2-7e7f5b64dbca admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:58:43.139340 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 884/1774] 10.222.0.33 () {58 vars in 1153 bytes} [Thu Sep 1 14:58:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:58:43.144729 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0505fc7e-651b-48a5-82ac-58c1df2d0cf6 None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:43.145561 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0505fc7e-651b-48a5-82ac-58c1df2d0cf6 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:43.145839 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0505fc7e-651b-48a5-82ac-58c1df2d0cf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:43.152508 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 891/1775] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 1 14:58:43 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:58:43.152863 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7876cc8d-7b05-4b36-92f2-7e7f5b64dbca admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:58:43.174657 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7876cc8d-7b05-4b36-92f2-7e7f5b64dbca admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:58:43.185098 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46acabf9-64c5-4e45-9cb9-6e56741d7b08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:58:43.192914 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46acabf9-64c5-4e45-9cb9-6e56741d7b08 admin admin] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:43.193251 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46acabf9-64c5-4e45-9cb9-6e56741d7b08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:43.193526 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46acabf9-64c5-4e45-9cb9-6e56741d7b08 admin admin] PATH_INFO: `/v3/projects` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:43.195139 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46acabf9-64c5-4e45-9cb9-6e56741d7b08 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:58:43.200668 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46acabf9-64c5-4e45-9cb9-6e56741d7b08 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:58:43.222583 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-0505fc7e-651b-48a5-82ac-58c1df2d0cf6 None None] MFA Rules not processed for user `2790149639aa418bbc4e4982e93eda44`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:58:43.228125 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-0505fc7e-651b-48a5-82ac-58c1df2d0cf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:58:43.281376 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:58:43.281585 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 892/1776] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Sep 1 14:58:43 2022] POST /identity/v3/projects => generated 389 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:58:43.282521 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46acabf9-64c5-4e45-9cb9-6e56741d7b08 admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:58:43.444575 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 885/1777] 10.222.0.33 () {58 vars in 1009 bytes} [Thu Sep 1 14:58:43 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 308 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:58:43.445728 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0505fc7e-651b-48a5-82ac-58c1df2d0cf6 None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:58:43.497542 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46acabf9-64c5-4e45-9cb9-6e56741d7b08 admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:58:43.500276 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0505fc7e-651b-48a5-82ac-58c1df2d0cf6 None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:58:43.512390 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed90926-78af-4d41-9648-5b7a4d3c5b36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:58:43.512612 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6087a17-efb5-482d-939f-7aef27c79369 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:58:43.513304 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6087a17-efb5-482d-939f-7aef27c79369 admin admin] REQUEST_METHOD: `PUT` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:43.513589 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6087a17-efb5-482d-939f-7aef27c79369 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:43.513848 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed90926-78af-4d41-9648-5b7a4d3c5b36 admin admin] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:43.513957 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6087a17-efb5-482d-939f-7aef27c79369 admin admin] PATH_INFO: `/v3/projects/56b842d4b732419f9df80f29475cc5a4/users/23dc5adbef02483d92026ca9f7d33925/roles/5d1feef0e13d4874928bd86258c4e3b2` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:43.514865 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed90926-78af-4d41-9648-5b7a4d3c5b36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:43.515149 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed90926-78af-4d41-9648-5b7a4d3c5b36 admin admin] PATH_INFO: `/v3/users` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:43.515794 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed90926-78af-4d41-9648-5b7a4d3c5b36 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:58:43.521467 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed90926-78af-4d41-9648-5b7a4d3c5b36 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:58:43.555146 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6087a17-efb5-482d-939f-7aef27c79369 admin admin] RBAC: Authorizing `identity:create_grant(project_id=56b842d4b732419f9df80f29475cc5a4, user_id=23dc5adbef02483d92026ca9f7d33925, role_id=5d1feef0e13d4874928bd86258c4e3b2)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:58:43.564065 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6087a17-efb5-482d-939f-7aef27c79369 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:58:43.576452 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 886/1778] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Sep 1 14:58:43 2022] POST /identity/v3/users => generated 295 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:58:43.577578 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed90926-78af-4d41-9648-5b7a4d3c5b36 admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:58:43.579761 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-eed90926-78af-4d41-9648-5b7a4d3c5b36 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:58:43.592316 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 893/1779] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 1 14:58:43 2022] PUT /identity/v3/projects/56b842d4b732419f9df80f29475cc5a4/users/23dc5adbef02483d92026ca9f7d33925/roles/5d1feef0e13d4874928bd86258c4e3b2 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:58:43.593498 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6087a17-efb5-482d-939f-7aef27c79369 admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:58:43.668866 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed90926-78af-4d41-9648-5b7a4d3c5b36 admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:58:43.722128 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:58:43.738209 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2e88ae-9d5a-4aa6-b9ed-4e719cd46262 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2790149639aa418bbc4e4982e93eda44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:58:43.739194 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2e88ae-9d5a-4aa6-b9ed-4e719cd46262 service neutron] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:43.739479 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2e88ae-9d5a-4aa6-b9ed-4e719cd46262 service neutron] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:43.739749 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2e88ae-9d5a-4aa6-b9ed-4e719cd46262 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:43.750387 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2e88ae-9d5a-4aa6-b9ed-4e719cd46262 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:58:43.752378 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2e88ae-9d5a-4aa6-b9ed-4e719cd46262 service neutron] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:58:43.765018 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 887/1780] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 14:58:43 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:58:43.766864 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee2203a-cddc-46e9-bed4-b57f12a792fc None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:43.767155 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee2203a-cddc-46e9-bed4-b57f12a792fc None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:43.767659 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee2203a-cddc-46e9-bed4-b57f12a792fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:43.818833 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-dee2203a-cddc-46e9-bed4-b57f12a792fc None None] MFA Rules not processed for user `23dc5adbef02483d92026ca9f7d33925`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:58:43.820254 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-dee2203a-cddc-46e9-bed4-b57f12a792fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:58:43.852876 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6087a17-efb5-482d-939f-7aef27c79369 admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:58:43.872386 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9e8370-3ec9-42a3-8aff-12a3fcd30501 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:58:43.877035 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9e8370-3ec9-42a3-8aff-12a3fcd30501 admin admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:43.877035 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9e8370-3ec9-42a3-8aff-12a3fcd30501 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:43.877035 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9e8370-3ec9-42a3-8aff-12a3fcd30501 admin admin] PATH_INFO: `/v3/roles` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:43.884840 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9e8370-3ec9-42a3-8aff-12a3fcd30501 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:58:43.893215 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9e8370-3ec9-42a3-8aff-12a3fcd30501 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:58:43.920456 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 888/1781] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 14:58:43 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:58:43.924462 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee2203a-cddc-46e9-bed4-b57f12a792fc None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:58:43.933705 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 894/1782] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 1 14:58:43 2022] GET /identity/v3/roles => generated 1420 bytes in 340 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:58:43.938867 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9e8370-3ec9-42a3-8aff-12a3fcd30501 admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:58:43.945757 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:58:43.952292 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee2203a-cddc-46e9-bed4-b57f12a792fc None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:58:43.958464 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baaa0863-25bd-4eb3-810d-d27376264547 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:58:43.959089 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaa0863-25bd-4eb3-810d-d27376264547 admin admin] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:43.959363 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaa0863-25bd-4eb3-810d-d27376264547 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:43.959631 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaa0863-25bd-4eb3-810d-d27376264547 admin admin] PATH_INFO: `/v3/projects` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:43.960276 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baaa0863-25bd-4eb3-810d-d27376264547 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:58:43.963294 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baaa0863-25bd-4eb3-810d-d27376264547 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:58:43.983858 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9e8370-3ec9-42a3-8aff-12a3fcd30501 admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:58:44.007880 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d9ab02-ddcf-4248-8f78-72330df3ead4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:58:44.008623 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d9ab02-ddcf-4248-8f78-72330df3ead4 admin admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:44.009068 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d9ab02-ddcf-4248-8f78-72330df3ead4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:44.009434 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d9ab02-ddcf-4248-8f78-72330df3ead4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:44.010276 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d9ab02-ddcf-4248-8f78-72330df3ead4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:58:44.014284 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d9ab02-ddcf-4248-8f78-72330df3ead4 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:58:44.023218 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 889/1783] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Sep 1 14:58:43 2022] POST /identity/v3/projects => generated 381 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:58:44.026824 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baaa0863-25bd-4eb3-810d-d27376264547 admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:58:44.058806 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 895/1784] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 1 14:58:43 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:58:44.059680 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d9ab02-ddcf-4248-8f78-72330df3ead4 admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:58:44.095082 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d9ab02-ddcf-4248-8f78-72330df3ead4 admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:58:44.231680 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baaa0863-25bd-4eb3-810d-d27376264547 admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:58:44.244864 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ed5b53-7a60-44bb-9fb7-3888617898be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:58:44.245468 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdef94a-73b4-4780-908c-5698fe6ca4de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:58:44.246279 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdef94a-73b4-4780-908c-5698fe6ca4de admin admin] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:44.246649 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdef94a-73b4-4780-908c-5698fe6ca4de admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:44.247040 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdef94a-73b4-4780-908c-5698fe6ca4de admin admin] PATH_INFO: `/v3/users` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:44.247800 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdef94a-73b4-4780-908c-5698fe6ca4de admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:58:44.247938 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ed5b53-7a60-44bb-9fb7-3888617898be admin admin] REQUEST_METHOD: `PUT` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:44.249800 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ed5b53-7a60-44bb-9fb7-3888617898be admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:44.250773 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ed5b53-7a60-44bb-9fb7-3888617898be admin admin] PATH_INFO: `/v3/projects/294bbe0323c64604b7e2caf1cd479a73/users/9fe359214d404a1fb3fa251bcf31402c/roles/5d1feef0e13d4874928bd86258c4e3b2` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:44.255651 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdef94a-73b4-4780-908c-5698fe6ca4de admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:58:44.334041 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ed5b53-7a60-44bb-9fb7-3888617898be admin admin] RBAC: Authorizing `identity:create_grant(project_id=294bbe0323c64604b7e2caf1cd479a73, user_id=9fe359214d404a1fb3fa251bcf31402c, role_id=5d1feef0e13d4874928bd86258c4e3b2)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:58:44.344451 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ed5b53-7a60-44bb-9fb7-3888617898be admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:58:44.344775 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 890/1785] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Sep 1 14:58:44 2022] POST /identity/v3/users => generated 297 bytes in 318 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:58:44.345855 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdef94a-73b4-4780-908c-5698fe6ca4de admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:58:44.371488 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdef94a-73b4-4780-908c-5698fe6ca4de admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:58:44.385937 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900798bf-321e-441b-98e9-086aae7978cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:58:44.386980 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900798bf-321e-441b-98e9-086aae7978cc admin admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:44.387490 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900798bf-321e-441b-98e9-086aae7978cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:44.387949 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900798bf-321e-441b-98e9-086aae7978cc admin admin] PATH_INFO: `/v3/roles` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:44.388856 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900798bf-321e-441b-98e9-086aae7978cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:58:44.392320 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900798bf-321e-441b-98e9-086aae7978cc admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:58:44.415377 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 891/1786] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 1 14:58:44 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:58:44.416523 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900798bf-321e-441b-98e9-086aae7978cc admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:58:44.430358 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 896/1787] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 1 14:58:44 2022] PUT /identity/v3/projects/294bbe0323c64604b7e2caf1cd479a73/users/9fe359214d404a1fb3fa251bcf31402c/roles/5d1feef0e13d4874928bd86258c4e3b2 => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:58:44.437213 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900798bf-321e-441b-98e9-086aae7978cc admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:58:44.443123 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bbe0cc-a885-443c-b014-264b81e6ff36 None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:44.443412 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bbe0cc-a885-443c-b014-264b81e6ff36 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:44.443688 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bbe0cc-a885-443c-b014-264b81e6ff36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:44.556265 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-64bbe0cc-a885-443c-b014-264b81e6ff36 None None] MFA Rules not processed for user `9fe359214d404a1fb3fa251bcf31402c`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:58:44.588105 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-64bbe0cc-a885-443c-b014-264b81e6ff36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:58:44.712983 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bac742-a8b8-42d3-9855-a6d1e8ddd996 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:58:44.716795 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bac742-a8b8-42d3-9855-a6d1e8ddd996 admin admin] REQUEST_METHOD: `PUT` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:44.717087 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bac742-a8b8-42d3-9855-a6d1e8ddd996 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:44.717367 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bac742-a8b8-42d3-9855-a6d1e8ddd996 admin admin] PATH_INFO: `/v3/projects/83167e938a2545b18956d1e86cf31680/users/f6afe0e86fec4f19addf70f46c664516/roles/2a2adb769e55476b82e5014897e0a9c0` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:44.772193 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01bac742-a8b8-42d3-9855-a6d1e8ddd996 admin admin] RBAC: Authorizing `identity:create_grant(project_id=83167e938a2545b18956d1e86cf31680, user_id=f6afe0e86fec4f19addf70f46c664516, role_id=2a2adb769e55476b82e5014897e0a9c0)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:58:44.774964 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01bac742-a8b8-42d3-9855-a6d1e8ddd996 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:58:44.834413 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 892/1788] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 1 14:58:44 2022] PUT /identity/v3/projects/83167e938a2545b18956d1e86cf31680/users/f6afe0e86fec4f19addf70f46c664516/roles/2a2adb769e55476b82e5014897e0a9c0 => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:58:44.837948 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bac742-a8b8-42d3-9855-a6d1e8ddd996 admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:58:44.912712 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 897/1789] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 14:58:44 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 472 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 14:58:44.917411 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bbe0cc-a885-443c-b014-264b81e6ff36 None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:58:44.969344 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bbe0cc-a885-443c-b014-264b81e6ff36 None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:58:44.985391 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:58:45.090376 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bac742-a8b8-42d3-9855-a6d1e8ddd996 admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:58:45.134506 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229b51ab-6a6e-4fe6-b67d-7e3f455594aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:58:45.137132 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229b51ab-6a6e-4fe6-b67d-7e3f455594aa admin admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:45.137416 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229b51ab-6a6e-4fe6-b67d-7e3f455594aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:45.137683 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229b51ab-6a6e-4fe6-b67d-7e3f455594aa admin admin] PATH_INFO: `/v3/roles` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:45.143837 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229b51ab-6a6e-4fe6-b67d-7e3f455594aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:58:45.145813 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229b51ab-6a6e-4fe6-b67d-7e3f455594aa admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:58:45.156998 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:58:45.173343 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 893/1790] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 1 14:58:44 2022] GET /identity/v3/roles => generated 1420 bytes in 335 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:58:45.186370 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229b51ab-6a6e-4fe6-b67d-7e3f455594aa admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:58:45.232622 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229b51ab-6a6e-4fe6-b67d-7e3f455594aa admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:58:45.245520 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951f3ee3-403e-437d-80d0-939dee3a9fc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:58:45.247709 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951f3ee3-403e-437d-80d0-939dee3a9fc1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:45.248016 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951f3ee3-403e-437d-80d0-939dee3a9fc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:45.248272 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951f3ee3-403e-437d-80d0-939dee3a9fc1 admin admin] PATH_INFO: `/v3/projects/83167e938a2545b18956d1e86cf31680/users/f6afe0e86fec4f19addf70f46c664516/roles/2a2adb769e55476b82e5014897e0a9c0` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:45.254074 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951f3ee3-403e-437d-80d0-939dee3a9fc1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=83167e938a2545b18956d1e86cf31680, user_id=f6afe0e86fec4f19addf70f46c664516, role_id=2a2adb769e55476b82e5014897e0a9c0)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:58:45.260468 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951f3ee3-403e-437d-80d0-939dee3a9fc1 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:58:45.299120 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701ee796-3872-4d82-b242-d7424ad7314b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:58:45.299120 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701ee796-3872-4d82-b242-d7424ad7314b admin admin] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:45.299120 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701ee796-3872-4d82-b242-d7424ad7314b admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:45.299120 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701ee796-3872-4d82-b242-d7424ad7314b admin admin] PATH_INFO: `/v3/projects` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:45.300192 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701ee796-3872-4d82-b242-d7424ad7314b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:58:45.312483 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701ee796-3872-4d82-b242-d7424ad7314b admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:58:45.325617 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 894/1791] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 1 14:58:45 2022] PUT /identity/v3/projects/83167e938a2545b18956d1e86cf31680/users/f6afe0e86fec4f19addf70f46c664516/roles/2a2adb769e55476b82e5014897e0a9c0 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:58:45.339807 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df201ead-a8f9-4e12-a680-1a68ebc45bc7 None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:45.340092 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df201ead-a8f9-4e12-a680-1a68ebc45bc7 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:45.340349 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df201ead-a8f9-4e12-a680-1a68ebc45bc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:45.457564 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 898/1792] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Sep 1 14:58:44 2022] POST /identity/v3/projects => generated 389 bytes in 540 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:58:45.459231 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701ee796-3872-4d82-b242-d7424ad7314b admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:58:45.485166 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-df201ead-a8f9-4e12-a680-1a68ebc45bc7 None None] MFA Rules not processed for user `f6afe0e86fec4f19addf70f46c664516`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:58:45.493834 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-df201ead-a8f9-4e12-a680-1a68ebc45bc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:58:45.716537 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701ee796-3872-4d82-b242-d7424ad7314b admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:58:45.723602 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8974f18a-a800-4a46-9f58-2d722606f571 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:58:45.724499 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8974f18a-a800-4a46-9f58-2d722606f571 admin admin] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:45.725175 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8974f18a-a800-4a46-9f58-2d722606f571 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:45.725620 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8974f18a-a800-4a46-9f58-2d722606f571 admin admin] PATH_INFO: `/v3/users` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:45.726778 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8974f18a-a800-4a46-9f58-2d722606f571 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:58:45.729875 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8974f18a-a800-4a46-9f58-2d722606f571 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:58:45.781930 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 895/1793] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 14:58:45 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 438 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:58:45.784748 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df201ead-a8f9-4e12-a680-1a68ebc45bc7 None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:58:45.792641 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 899/1794] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Sep 1 14:58:45 2022] POST /identity/v3/users => generated 301 bytes in 334 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:58:45.793905 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8974f18a-a800-4a46-9f58-2d722606f571 admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:58:45.800215 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-8974f18a-a800-4a46-9f58-2d722606f571 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:58:45.816524 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df201ead-a8f9-4e12-a680-1a68ebc45bc7 None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:58:45.837103 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7635658b-a632-43b4-ac4e-cdebc9992dbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:58:45.837716 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7635658b-a632-43b4-ac4e-cdebc9992dbb admin admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:45.838001 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7635658b-a632-43b4-ac4e-cdebc9992dbb admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:45.838307 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7635658b-a632-43b4-ac4e-cdebc9992dbb admin admin] PATH_INFO: `/v3/roles` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:45.838985 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7635658b-a632-43b4-ac4e-cdebc9992dbb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:58:45.841851 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7635658b-a632-43b4-ac4e-cdebc9992dbb admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:58:45.869664 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 896/1795] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 1 14:58:45 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:58:45.873394 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7635658b-a632-43b4-ac4e-cdebc9992dbb admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:58:45.936680 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7635658b-a632-43b4-ac4e-cdebc9992dbb admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:58:45.963052 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083d025e-2946-42ac-8ea8-424c05c9290c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:58:45.963655 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083d025e-2946-42ac-8ea8-424c05c9290c admin admin] REQUEST_METHOD: `PUT` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:45.964045 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083d025e-2946-42ac-8ea8-424c05c9290c admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:45.964325 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083d025e-2946-42ac-8ea8-424c05c9290c admin admin] PATH_INFO: `/v3/projects/7854e13fd33c40f19590afb99239f6eb/users/273526ce633c48deb1000ce1d4029ea4/roles/2a2adb769e55476b82e5014897e0a9c0` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:45.997148 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083d025e-2946-42ac-8ea8-424c05c9290c admin admin] RBAC: Authorizing `identity:create_grant(project_id=7854e13fd33c40f19590afb99239f6eb, user_id=273526ce633c48deb1000ce1d4029ea4, role_id=2a2adb769e55476b82e5014897e0a9c0)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:58:46.001030 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083d025e-2946-42ac-8ea8-424c05c9290c admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:58:46.027258 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 897/1796] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 1 14:58:45 2022] PUT /identity/v3/projects/7854e13fd33c40f19590afb99239f6eb/users/273526ce633c48deb1000ce1d4029ea4/roles/2a2adb769e55476b82e5014897e0a9c0 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:58:46.031093 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083d025e-2946-42ac-8ea8-424c05c9290c admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:58:46.058215 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8974f18a-a800-4a46-9f58-2d722606f571 admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:58:46.238669 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd93a19b-6cd7-4395-82f8-5e451f9a60a3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3b81739e34e4e648f88b6d73fcb5e46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:58:46.239357 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd93a19b-6cd7-4395-82f8-5e451f9a60a3 service cinder] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:46.239614 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd93a19b-6cd7-4395-82f8-5e451f9a60a3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:46.239895 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd93a19b-6cd7-4395-82f8-5e451f9a60a3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:46.270638 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:58:46.306681 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083d025e-2946-42ac-8ea8-424c05c9290c admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:58:46.321089 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5632bcb-8f07-4522-be63-f392a4af30c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:58:46.321789 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5632bcb-8f07-4522-be63-f392a4af30c1 admin admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:46.322751 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5632bcb-8f07-4522-be63-f392a4af30c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:46.323102 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5632bcb-8f07-4522-be63-f392a4af30c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:46.324201 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5632bcb-8f07-4522-be63-f392a4af30c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:58:46.330155 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5632bcb-8f07-4522-be63-f392a4af30c1 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:58:46.357735 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 898/1797] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 1 14:58:46 2022] GET /identity/v3/roles => generated 1420 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 14:58:46.361477 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5632bcb-8f07-4522-be63-f392a4af30c1 admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:58:46.362589 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd93a19b-6cd7-4395-82f8-5e451f9a60a3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:58:46.364178 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd93a19b-6cd7-4395-82f8-5e451f9a60a3 service cinder] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:58:46.380723 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 900/1798] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 14:58:45 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 587 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:58:46.383003 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5632bcb-8f07-4522-be63-f392a4af30c1 admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:58:46.403033 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bd07b2-826f-4497-aceb-25fb171add6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:58:46.446893 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bd07b2-826f-4497-aceb-25fb171add6c admin admin] REQUEST_METHOD: `PUT` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:46.451187 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bd07b2-826f-4497-aceb-25fb171add6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:46.451538 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bd07b2-826f-4497-aceb-25fb171add6c admin admin] PATH_INFO: `/v3/projects/7854e13fd33c40f19590afb99239f6eb/users/273526ce633c48deb1000ce1d4029ea4/roles/2a2adb769e55476b82e5014897e0a9c0` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:46.459396 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bd07b2-826f-4497-aceb-25fb171add6c admin admin] RBAC: Authorizing `identity:create_grant(project_id=7854e13fd33c40f19590afb99239f6eb, user_id=273526ce633c48deb1000ce1d4029ea4, role_id=2a2adb769e55476b82e5014897e0a9c0)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:58:46.485096 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bd07b2-826f-4497-aceb-25fb171add6c admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:58:46.513815 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:58:46.534374 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 899/1799] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 1 14:58:46 2022] PUT /identity/v3/projects/7854e13fd33c40f19590afb99239f6eb/users/273526ce633c48deb1000ce1d4029ea4/roles/2a2adb769e55476b82e5014897e0a9c0 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 14:58:46.550220 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd594dc6-49cc-4ee8-8a3c-2cf212d7a972 None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:46.550220 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd594dc6-49cc-4ee8-8a3c-2cf212d7a972 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:46.550220 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd594dc6-49cc-4ee8-8a3c-2cf212d7a972 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:46.684631 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-fd594dc6-49cc-4ee8-8a3c-2cf212d7a972 None None] MFA Rules not processed for user `273526ce633c48deb1000ce1d4029ea4`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:58:46.693768 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-fd594dc6-49cc-4ee8-8a3c-2cf212d7a972 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:58:46.745144 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bd07b2-826f-4497-aceb-25fb171add6c admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:58:46.907685 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bd07b2-826f-4497-aceb-25fb171add6c admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:58:46.918447 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4682658-5d5d-4ebd-b57d-d2a4dde7a7fa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3b81739e34e4e648f88b6d73fcb5e46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:58:46.920427 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4682658-5d5d-4ebd-b57d-d2a4dde7a7fa service cinder] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:46.920821 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4682658-5d5d-4ebd-b57d-d2a4dde7a7fa service cinder] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:46.921810 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4682658-5d5d-4ebd-b57d-d2a4dde7a7fa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:46.976200 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 901/1800] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 14:58:46 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 431 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:58:47.154273 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4682658-5d5d-4ebd-b57d-d2a4dde7a7fa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:58:47.156269 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4682658-5d5d-4ebd-b57d-d2a4dde7a7fa service cinder] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:58:47.199307 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 900/1801] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 14:58:46 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 455 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:58:47.256040 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59143a3-bf9a-4c69-b352-a7c440630fc7 None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:47.256588 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59143a3-bf9a-4c69-b352-a7c440630fc7 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:47.256819 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59143a3-bf9a-4c69-b352-a7c440630fc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:47.307164 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-b59143a3-bf9a-4c69-b352-a7c440630fc7 None None] MFA Rules not processed for user `24b6f275ffb04ffeae3c2097bfd040f5`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:58:47.308535 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-b59143a3-bf9a-4c69-b352-a7c440630fc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:58:47.315018 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4682658-5d5d-4ebd-b57d-d2a4dde7a7fa service cinder] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:58:47.316717 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-a4682658-5d5d-4ebd-b57d-d2a4dde7a7fa service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:58:47.336207 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4682658-5d5d-4ebd-b57d-d2a4dde7a7fa service cinder] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:58:47.344170 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6831e3-4397-4bfa-bae6-9ebdd165c04b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3b81739e34e4e648f88b6d73fcb5e46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:58:47.350018 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6831e3-4397-4bfa-bae6-9ebdd165c04b service cinder] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:47.350530 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6831e3-4397-4bfa-bae6-9ebdd165c04b service cinder] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:47.351716 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6831e3-4397-4bfa-bae6-9ebdd165c04b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:47.381192 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6831e3-4397-4bfa-bae6-9ebdd165c04b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:58:47.385647 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6831e3-4397-4bfa-bae6-9ebdd165c04b service cinder] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:58:47.445684 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 901/1802] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 14:58:47 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:58:47.493135 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 902/1803] 10.222.0.33 () {58 vars in 1009 bytes} [Thu Sep 1 14:58:47 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:58:47.509386 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6831e3-4397-4bfa-bae6-9ebdd165c04b service cinder] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:58:47.510425 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-7e6831e3-4397-4bfa-bae6-9ebdd165c04b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:58:47.657027 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6831e3-4397-4bfa-bae6-9ebdd165c04b service cinder] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:58:47.676086 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ac5697-7d34-4c82-bc17-e065ab7c5048 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73a5bda5a6ea46e9863ccafd57dee339', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:58:47.677029 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ac5697-7d34-4c82-bc17-e065ab7c5048 service swift] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:47.678847 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ac5697-7d34-4c82-bc17-e065ab7c5048 service swift] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:47.679454 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ac5697-7d34-4c82-bc17-e065ab7c5048 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:47.702186 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ac5697-7d34-4c82-bc17-e065ab7c5048 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:58:47.704592 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ac5697-7d34-4c82-bc17-e065ab7c5048 service swift] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:58:47.767270 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 902/1804] 10.222.0.33 () {60 vars in 1384 bytes} [Thu Sep 1 14:58:47 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 260 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 01 14:58:48.203752 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59143a3-bf9a-4c69-b352-a7c440630fc7 None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:58:48.386696 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:58:48.453757 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59143a3-bf9a-4c69-b352-a7c440630fc7 None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:58:48.477079 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb288465-28f2-4f65-825b-b92d0c091fe4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:58:48.477897 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb288465-28f2-4f65-825b-b92d0c091fe4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:48.478321 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb288465-28f2-4f65-825b-b92d0c091fe4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:48.478663 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb288465-28f2-4f65-825b-b92d0c091fe4 admin admin] PATH_INFO: `/v3/users/9fe359214d404a1fb3fa251bcf31402c` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:48.482151 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb288465-28f2-4f65-825b-b92d0c091fe4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9fe359214d404a1fb3fa251bcf31402c)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:58:48.491180 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb288465-28f2-4f65-825b-b92d0c091fe4 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:58:48.652170 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-bb288465-28f2-4f65-825b-b92d0c091fe4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9fe359214d404a1fb3fa251bcf31402c'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:58:48.681042 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-bb288465-28f2-4f65-825b-b92d0c091fe4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9fe359214d404a1fb3fa251bcf31402c'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:58:48.682660 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-bb288465-28f2-4f65-825b-b92d0c091fe4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9fe359214d404a1fb3fa251bcf31402c'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:58:48.696958 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ac5697-7d34-4c82-bc17-e065ab7c5048 service swift] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:58:48.715451 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ac5697-7d34-4c82-bc17-e065ab7c5048 service swift] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:58:48.727864 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-bb288465-28f2-4f65-825b-b92d0c091fe4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9fe359214d404a1fb3fa251bcf31402c'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:58:48.729121 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-bb288465-28f2-4f65-825b-b92d0c091fe4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9fe359214d404a1fb3fa251bcf31402c'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:58:48.732329 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991b3ea6-7ec8-4cc5-8a74-6c993519f64c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:58:48.751640 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991b3ea6-7ec8-4cc5-8a74-6c993519f64c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:48.751931 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991b3ea6-7ec8-4cc5-8a74-6c993519f64c admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:48.752210 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991b3ea6-7ec8-4cc5-8a74-6c993519f64c admin admin] PATH_INFO: `/v3/users/23dc5adbef02483d92026ca9f7d33925` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:48.765373 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991b3ea6-7ec8-4cc5-8a74-6c993519f64c admin admin] RBAC: Authorizing `identity:delete_user(user_id=23dc5adbef02483d92026ca9f7d33925)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:58:48.780312 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991b3ea6-7ec8-4cc5-8a74-6c993519f64c admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:58:48.789435 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 903/1805] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Sep 1 14:58:48 2022] DELETE /identity/v3/users/9fe359214d404a1fb3fa251bcf31402c => generated 0 bytes in 586 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:58:48.800705 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb35681-263d-4df1-a2c2-d64887ce2453 None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:48.803543 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb35681-263d-4df1-a2c2-d64887ce2453 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:48.804352 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb35681-263d-4df1-a2c2-d64887ce2453 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:48.893157 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:58:48.942217 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-ebb35681-263d-4df1-a2c2-d64887ce2453 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:58:48.953458 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-ebb35681-263d-4df1-a2c2-d64887ce2453 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:58:48.989588 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-991b3ea6-7ec8-4cc5-8a74-6c993519f64c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '23dc5adbef02483d92026ca9f7d33925'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:58:49.009550 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-991b3ea6-7ec8-4cc5-8a74-6c993519f64c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '23dc5adbef02483d92026ca9f7d33925'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:58:49.010914 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-991b3ea6-7ec8-4cc5-8a74-6c993519f64c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '23dc5adbef02483d92026ca9f7d33925'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:58:49.124209 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 904/1806] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 14:58:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 324 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:58:49.137114 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-991b3ea6-7ec8-4cc5-8a74-6c993519f64c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '23dc5adbef02483d92026ca9f7d33925'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:58:49.138615 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-991b3ea6-7ec8-4cc5-8a74-6c993519f64c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '23dc5adbef02483d92026ca9f7d33925'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:58:49.142589 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb35681-263d-4df1-a2c2-d64887ce2453 None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:58:49.147469 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-ebb35681-263d-4df1-a2c2-d64887ce2453 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:58:49.206572 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 903/1807] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Sep 1 14:58:48 2022] DELETE /identity/v3/users/23dc5adbef02483d92026ca9f7d33925 => generated 0 bytes in 510 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 14:58:49.211808 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d55c80-6a2a-42fe-9849-a993a5b0db3a None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:49.212141 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d55c80-6a2a-42fe-9849-a993a5b0db3a None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:49.212418 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d55c80-6a2a-42fe-9849-a993a5b0db3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:49.298490 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-97d55c80-6a2a-42fe-9849-a993a5b0db3a None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:58:49.308235 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-97d55c80-6a2a-42fe-9849-a993a5b0db3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:58:49.335089 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb35681-263d-4df1-a2c2-d64887ce2453 None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:58:49.399510 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:58:49.534974 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 904/1808] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 14:58:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 324 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:58:49.541243 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0a5a64-8f64-4f56-991c-e3ecaa0ea7f8 None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:49.541536 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0a5a64-8f64-4f56-991c-e3ecaa0ea7f8 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:49.541816 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0a5a64-8f64-4f56-991c-e3ecaa0ea7f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:49.578013 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b062342-21fa-48e3-9deb-f657d416cdf3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2790149639aa418bbc4e4982e93eda44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:58:49.578839 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b062342-21fa-48e3-9deb-f657d416cdf3 service neutron] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:49.579012 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b062342-21fa-48e3-9deb-f657d416cdf3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:49.579274 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b062342-21fa-48e3-9deb-f657d416cdf3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:49.588878 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-0b062342-21fa-48e3-9deb-f657d416cdf3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:58:49.612670 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-ec0a5a64-8f64-4f56-991c-e3ecaa0ea7f8 None None] MFA Rules not processed for user `82255a5060674d77a554e34f89ab1bff`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:58:49.617360 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-ec0a5a64-8f64-4f56-991c-e3ecaa0ea7f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:58:49.636757 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b062342-21fa-48e3-9deb-f657d416cdf3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:58:49.638381 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b062342-21fa-48e3-9deb-f657d416cdf3 service neutron] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:58:49.699226 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 905/1809] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 14:58:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 557 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:58:49.700252 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b062342-21fa-48e3-9deb-f657d416cdf3 service neutron] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:58:49.709858 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-0b062342-21fa-48e3-9deb-f657d416cdf3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:58:49.737928 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b062342-21fa-48e3-9deb-f657d416cdf3 service neutron] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:58:49.752580 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8447e3bf-a7a0-4b83-b21d-b99a081de1e8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2790149639aa418bbc4e4982e93eda44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:58:49.753591 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8447e3bf-a7a0-4b83-b21d-b99a081de1e8 service neutron] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:49.756717 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8447e3bf-a7a0-4b83-b21d-b99a081de1e8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:49.757124 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8447e3bf-a7a0-4b83-b21d-b99a081de1e8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:49.790940 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8447e3bf-a7a0-4b83-b21d-b99a081de1e8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:58:49.794203 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8447e3bf-a7a0-4b83-b21d-b99a081de1e8 service neutron] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:58:49.807683 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 905/1810] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 14:58:49 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:58:49.817402 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 906/1811] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 14:58:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:58:50.882429 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0a5a64-8f64-4f56-991c-e3ecaa0ea7f8 None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:58:50.884827 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-ec0a5a64-8f64-4f56-991c-e3ecaa0ea7f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:58:50.930303 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0a5a64-8f64-4f56-991c-e3ecaa0ea7f8 None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:58:50.970299 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f6e8ef-3a69-4ddb-bc92-02dc67ce2ac7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:58:50.971209 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f6e8ef-3a69-4ddb-bc92-02dc67ce2ac7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:50.972307 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f6e8ef-3a69-4ddb-bc92-02dc67ce2ac7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:50.977223 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f6e8ef-3a69-4ddb-bc92-02dc67ce2ac7 admin admin] PATH_INFO: `/v3/projects/294bbe0323c64604b7e2caf1cd479a73` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:50.986949 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f6e8ef-3a69-4ddb-bc92-02dc67ce2ac7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=294bbe0323c64604b7e2caf1cd479a73)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:58:50.993918 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f6e8ef-3a69-4ddb-bc92-02dc67ce2ac7 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:58:51.118744 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:58:51.128723 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-a2f6e8ef-3a69-4ddb-bc92-02dc67ce2ac7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '294bbe0323c64604b7e2caf1cd479a73', 'request_id': 'req-a2f6e8ef-3a69-4ddb-bc92-02dc67ce2ac7'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:58:51.142449 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-a2f6e8ef-3a69-4ddb-bc92-02dc67ce2ac7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '294bbe0323c64604b7e2caf1cd479a73', 'request_id': 'req-a2f6e8ef-3a69-4ddb-bc92-02dc67ce2ac7'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:58:51.156852 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-a2f6e8ef-3a69-4ddb-bc92-02dc67ce2ac7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '294bbe0323c64604b7e2caf1cd479a73', 'request_id': 'req-a2f6e8ef-3a69-4ddb-bc92-02dc67ce2ac7'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:58:51.185088 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8447e3bf-a7a0-4b83-b21d-b99a081de1e8 service neutron] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:58:51.187870 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-8447e3bf-a7a0-4b83-b21d-b99a081de1e8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:58:51.198509 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-a2f6e8ef-3a69-4ddb-bc92-02dc67ce2ac7 admin admin] The token cache is being invalidate because project 294bbe0323c64604b7e2caf1cd479a73 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81341) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:58:51.211895 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-a2f6e8ef-3a69-4ddb-bc92-02dc67ce2ac7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:58:51.213831 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-a2f6e8ef-3a69-4ddb-bc92-02dc67ce2ac7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:58:51.221442 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 906/1812] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Sep 1 14:58:50 2022] DELETE /identity/v3/projects/294bbe0323c64604b7e2caf1cd479a73 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:58:51.226394 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f6e8ef-3a69-4ddb-bc92-02dc67ce2ac7 admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:58:51.229198 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-a2f6e8ef-3a69-4ddb-bc92-02dc67ce2ac7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:58:51.240059 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8447e3bf-a7a0-4b83-b21d-b99a081de1e8 service neutron] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:58:51.456472 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d05604-72ac-44b1-a9d1-c284f76d855d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:58:51.460136 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d05604-72ac-44b1-a9d1-c284f76d855d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:51.464139 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d05604-72ac-44b1-a9d1-c284f76d855d admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:51.465609 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d05604-72ac-44b1-a9d1-c284f76d855d admin admin] PATH_INFO: `/v3/projects/56b842d4b732419f9df80f29475cc5a4` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:51.470169 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d05604-72ac-44b1-a9d1-c284f76d855d admin admin] RBAC: Authorizing `identity:delete_project(project_id=56b842d4b732419f9df80f29475cc5a4)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:58:51.477456 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f6e8ef-3a69-4ddb-bc92-02dc67ce2ac7 admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:58:51.480113 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d05604-72ac-44b1-a9d1-c284f76d855d admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:58:51.494840 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426a1c5e-47b4-468b-a70e-4b235c8bcea6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:58:51.497749 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426a1c5e-47b4-468b-a70e-4b235c8bcea6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:51.502546 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426a1c5e-47b4-468b-a70e-4b235c8bcea6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:51.502803 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426a1c5e-47b4-468b-a70e-4b235c8bcea6 admin admin] PATH_INFO: `/v3/users/273526ce633c48deb1000ce1d4029ea4` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:51.505241 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426a1c5e-47b4-468b-a70e-4b235c8bcea6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=273526ce633c48deb1000ce1d4029ea4)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:58:51.511537 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426a1c5e-47b4-468b-a70e-4b235c8bcea6 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:58:51.683642 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-f5d05604-72ac-44b1-a9d1-c284f76d855d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '56b842d4b732419f9df80f29475cc5a4', 'request_id': 'req-f5d05604-72ac-44b1-a9d1-c284f76d855d'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:58:51.716221 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-426a1c5e-47b4-468b-a70e-4b235c8bcea6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '273526ce633c48deb1000ce1d4029ea4'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:58:51.726068 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-f5d05604-72ac-44b1-a9d1-c284f76d855d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '56b842d4b732419f9df80f29475cc5a4', 'request_id': 'req-f5d05604-72ac-44b1-a9d1-c284f76d855d'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:58:51.767694 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-426a1c5e-47b4-468b-a70e-4b235c8bcea6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '273526ce633c48deb1000ce1d4029ea4'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:58:51.768847 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-426a1c5e-47b4-468b-a70e-4b235c8bcea6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '273526ce633c48deb1000ce1d4029ea4'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:58:51.775190 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-f5d05604-72ac-44b1-a9d1-c284f76d855d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '56b842d4b732419f9df80f29475cc5a4', 'request_id': 'req-f5d05604-72ac-44b1-a9d1-c284f76d855d'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:58:51.855488 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-f5d05604-72ac-44b1-a9d1-c284f76d855d admin admin] The token cache is being invalidate because project 56b842d4b732419f9df80f29475cc5a4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81342) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:58:51.855832 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-f5d05604-72ac-44b1-a9d1-c284f76d855d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:58:51.860248 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-426a1c5e-47b4-468b-a70e-4b235c8bcea6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '273526ce633c48deb1000ce1d4029ea4'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:58:51.860646 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-f5d05604-72ac-44b1-a9d1-c284f76d855d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:58:51.863204 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-426a1c5e-47b4-468b-a70e-4b235c8bcea6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '273526ce633c48deb1000ce1d4029ea4'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:58:51.867746 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 907/1813] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Sep 1 14:58:51 2022] DELETE /identity/v3/projects/56b842d4b732419f9df80f29475cc5a4 => generated 0 bytes in 684 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:58:51.869051 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5a929a-3d42-4e16-a412-13d5942d6011 None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:51.869333 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5a929a-3d42-4e16-a412-13d5942d6011 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:51.869617 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5a929a-3d42-4e16-a412-13d5942d6011 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:51.945013 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 907/1814] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Sep 1 14:58:51 2022] DELETE /identity/v3/users/273526ce633c48deb1000ce1d4029ea4 => generated 0 bytes in 717 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:58:51.949220 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426a1c5e-47b4-468b-a70e-4b235c8bcea6 admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:58:51.959393 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-426a1c5e-47b4-468b-a70e-4b235c8bcea6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:58:51.988177 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-ef5a929a-3d42-4e16-a412-13d5942d6011 None None] MFA Rules not processed for user `2790149639aa418bbc4e4982e93eda44`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:58:51.991224 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-ef5a929a-3d42-4e16-a412-13d5942d6011 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:58:52.139483 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:58:52.189427 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:58:52.218089 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 908/1815] 10.222.0.65 () {58 vars in 1015 bytes} [Thu Sep 1 14:58:51 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 350 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:58:52.275801 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426a1c5e-47b4-468b-a70e-4b235c8bcea6 admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:58:52.293215 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187f8425-55a6-40c3-8085-2f4962a73e64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:58:52.293664 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187f8425-55a6-40c3-8085-2f4962a73e64 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:52.293930 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187f8425-55a6-40c3-8085-2f4962a73e64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:52.294168 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187f8425-55a6-40c3-8085-2f4962a73e64 admin admin] PATH_INFO: `/v3/users/f6afe0e86fec4f19addf70f46c664516` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:52.295535 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187f8425-55a6-40c3-8085-2f4962a73e64 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f6afe0e86fec4f19addf70f46c664516)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:58:52.297669 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187f8425-55a6-40c3-8085-2f4962a73e64 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:58:52.302129 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5a929a-3d42-4e16-a412-13d5942d6011 None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:58:52.304089 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-ef5a929a-3d42-4e16-a412-13d5942d6011 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:58:52.335453 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5a929a-3d42-4e16-a412-13d5942d6011 None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:58:52.375455 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543a5f09-ca30-48a5-9eb4-fb33a8563386 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2790149639aa418bbc4e4982e93eda44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:58:52.378767 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543a5f09-ca30-48a5-9eb4-fb33a8563386 service neutron] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:52.382111 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543a5f09-ca30-48a5-9eb4-fb33a8563386 service neutron] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:52.387284 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543a5f09-ca30-48a5-9eb4-fb33a8563386 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:52.406659 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-187f8425-55a6-40c3-8085-2f4962a73e64 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f6afe0e86fec4f19addf70f46c664516'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:58:52.425712 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-187f8425-55a6-40c3-8085-2f4962a73e64 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f6afe0e86fec4f19addf70f46c664516'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:58:52.428558 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-187f8425-55a6-40c3-8085-2f4962a73e64 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f6afe0e86fec4f19addf70f46c664516'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:58:52.443564 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543a5f09-ca30-48a5-9eb4-fb33a8563386 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:58:52.447437 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543a5f09-ca30-48a5-9eb4-fb33a8563386 service neutron] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:58:52.469019 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-187f8425-55a6-40c3-8085-2f4962a73e64 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f6afe0e86fec4f19addf70f46c664516'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:58:52.472789 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-187f8425-55a6-40c3-8085-2f4962a73e64 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f6afe0e86fec4f19addf70f46c664516'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:58:52.478783 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 909/1816] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 14:58:52 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 14:58:52.524328 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 908/1817] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Sep 1 14:58:51 2022] DELETE /identity/v3/users/f6afe0e86fec4f19addf70f46c664516 => generated 0 bytes in 579 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:58:54.357223 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543a5f09-ca30-48a5-9eb4-fb33a8563386 service neutron] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:58:54.382292 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543a5f09-ca30-48a5-9eb4-fb33a8563386 service neutron] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:58:54.525077 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80abb714-78ba-44b9-a02c-f810b13479e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:58:54.525998 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80abb714-78ba-44b9-a02c-f810b13479e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:54.526621 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80abb714-78ba-44b9-a02c-f810b13479e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:54.527163 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80abb714-78ba-44b9-a02c-f810b13479e4 admin admin] PATH_INFO: `/v3/projects/83167e938a2545b18956d1e86cf31680` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:54.529115 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80abb714-78ba-44b9-a02c-f810b13479e4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=83167e938a2545b18956d1e86cf31680)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:58:54.531440 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80abb714-78ba-44b9-a02c-f810b13479e4 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:58:54.708491 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-80abb714-78ba-44b9-a02c-f810b13479e4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '83167e938a2545b18956d1e86cf31680', 'request_id': 'req-80abb714-78ba-44b9-a02c-f810b13479e4'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:58:54.730228 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-80abb714-78ba-44b9-a02c-f810b13479e4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '83167e938a2545b18956d1e86cf31680', 'request_id': 'req-80abb714-78ba-44b9-a02c-f810b13479e4'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:58:54.750998 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-80abb714-78ba-44b9-a02c-f810b13479e4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '83167e938a2545b18956d1e86cf31680', 'request_id': 'req-80abb714-78ba-44b9-a02c-f810b13479e4'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:58:54.778886 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-80abb714-78ba-44b9-a02c-f810b13479e4 admin admin] The token cache is being invalidate because project 83167e938a2545b18956d1e86cf31680 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81342) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:58:54.781570 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-80abb714-78ba-44b9-a02c-f810b13479e4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:58:54.784108 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-80abb714-78ba-44b9-a02c-f810b13479e4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:58:54.788962 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 910/1818] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Sep 1 14:58:54 2022] DELETE /identity/v3/projects/83167e938a2545b18956d1e86cf31680 => generated 0 bytes in 432 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:58:54.795197 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84888541-55a9-4ded-9f0f-617a9f1d1c79 None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:54.795752 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84888541-55a9-4ded-9f0f-617a9f1d1c79 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:54.796064 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84888541-55a9-4ded-9f0f-617a9f1d1c79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:54.842863 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-84888541-55a9-4ded-9f0f-617a9f1d1c79 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:58:54.845365 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-84888541-55a9-4ded-9f0f-617a9f1d1c79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:58:54.964242 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 909/1819] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 14:58:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:58:54.973185 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab2885d-408d-4e97-8704-d14471ada556 None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:54.973594 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab2885d-408d-4e97-8704-d14471ada556 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:54.973986 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab2885d-408d-4e97-8704-d14471ada556 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:55.013054 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-5ab2885d-408d-4e97-8704-d14471ada556 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:58:55.018665 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-5ab2885d-408d-4e97-8704-d14471ada556 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:58:55.038602 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 911/1820] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 14:58:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:58:55.041579 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84888541-55a9-4ded-9f0f-617a9f1d1c79 None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:58:55.054999 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84888541-55a9-4ded-9f0f-617a9f1d1c79 None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:58:55.063182 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97bd370-cf34-4063-a459-06e1d626759d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:58:55.064546 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97bd370-cf34-4063-a459-06e1d626759d admin admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:55.064920 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97bd370-cf34-4063-a459-06e1d626759d admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:55.065299 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97bd370-cf34-4063-a459-06e1d626759d admin admin] PATH_INFO: `/v3/domains` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:55.066262 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97bd370-cf34-4063-a459-06e1d626759d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81341) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:58:55.067232 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97bd370-cf34-4063-a459-06e1d626759d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:58:55.069250 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97bd370-cf34-4063-a459-06e1d626759d admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:58:55.099050 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 910/1821] 10.222.0.33 () {58 vars in 1153 bytes} [Thu Sep 1 14:58:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:58:55.105864 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab2885d-408d-4e97-8704-d14471ada556 None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:58:55.137744 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab2885d-408d-4e97-8704-d14471ada556 None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:58:55.155574 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bce5df-1aea-46dd-847c-cfb3f97eb436 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:58:55.157601 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bce5df-1aea-46dd-847c-cfb3f97eb436 admin admin] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:55.159339 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bce5df-1aea-46dd-847c-cfb3f97eb436 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:55.160432 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bce5df-1aea-46dd-847c-cfb3f97eb436 admin admin] PATH_INFO: `/v3/projects` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:55.164169 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1bce5df-1aea-46dd-847c-cfb3f97eb436 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:58:55.169069 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1bce5df-1aea-46dd-847c-cfb3f97eb436 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:58:55.282392 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 912/1822] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Sep 1 14:58:55 2022] POST /identity/v3/projects => generated 375 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:58:55.287211 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97bd370-cf34-4063-a459-06e1d626759d admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:58:55.301987 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97bd370-cf34-4063-a459-06e1d626759d admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:58:55.419531 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:58:55.432115 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c54d870-7e2d-43cf-aeec-c28a69839666 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:58:55.433006 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c54d870-7e2d-43cf-aeec-c28a69839666 admin admin] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:55.435109 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c54d870-7e2d-43cf-aeec-c28a69839666 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:55.435290 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c54d870-7e2d-43cf-aeec-c28a69839666 admin admin] PATH_INFO: `/v3/users` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:55.435947 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c54d870-7e2d-43cf-aeec-c28a69839666 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:58:55.437968 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c54d870-7e2d-43cf-aeec-c28a69839666 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:58:55.483319 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 911/1823] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Sep 1 14:58:55 2022] POST /identity/v3/users => generated 288 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:58:55.484524 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bce5df-1aea-46dd-847c-cfb3f97eb436 admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:58:55.499134 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bce5df-1aea-46dd-847c-cfb3f97eb436 admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:58:55.513853 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55762404-15c6-4da4-96df-f208e21484b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:58:55.514549 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55762404-15c6-4da4-96df-f208e21484b1 admin admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:55.514775 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55762404-15c6-4da4-96df-f208e21484b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:55.515060 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55762404-15c6-4da4-96df-f208e21484b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:55.515762 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55762404-15c6-4da4-96df-f208e21484b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:58:55.517490 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55762404-15c6-4da4-96df-f208e21484b1 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:58:55.542652 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c54d870-7e2d-43cf-aeec-c28a69839666 admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:58:55.552728 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 913/1824] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 1 14:58:55 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:58:55.555048 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c54d870-7e2d-43cf-aeec-c28a69839666 admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:58:55.570850 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade3b143-83c3-425a-8b04-717cb539043d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:58:55.571328 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade3b143-83c3-425a-8b04-717cb539043d admin admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:55.571608 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade3b143-83c3-425a-8b04-717cb539043d admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:55.571879 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade3b143-83c3-425a-8b04-717cb539043d admin admin] PATH_INFO: `/v3/roles` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:55.572632 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade3b143-83c3-425a-8b04-717cb539043d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:58:55.576916 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade3b143-83c3-425a-8b04-717cb539043d admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:58:55.620175 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 912/1825] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 1 14:58:55 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:58:55.621098 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55762404-15c6-4da4-96df-f208e21484b1 admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:58:55.636694 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55762404-15c6-4da4-96df-f208e21484b1 admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:58:55.656417 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a60a459-4f5d-43f9-a379-1df12a5bbf2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:58:55.657110 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a60a459-4f5d-43f9-a379-1df12a5bbf2d admin admin] REQUEST_METHOD: `PUT` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:55.657399 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a60a459-4f5d-43f9-a379-1df12a5bbf2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:55.657679 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a60a459-4f5d-43f9-a379-1df12a5bbf2d admin admin] PATH_INFO: `/v3/projects/2822fc0546b94a28a0dbb37aee02f124/users/8c5ccfb591d14333b986fd0b9a3cf3be/roles/5d1feef0e13d4874928bd86258c4e3b2` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:55.714322 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a60a459-4f5d-43f9-a379-1df12a5bbf2d admin admin] RBAC: Authorizing `identity:create_grant(project_id=2822fc0546b94a28a0dbb37aee02f124, user_id=8c5ccfb591d14333b986fd0b9a3cf3be, role_id=5d1feef0e13d4874928bd86258c4e3b2)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:58:55.736046 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a60a459-4f5d-43f9-a379-1df12a5bbf2d admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:58:55.758741 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:58:55.775173 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 914/1826] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 1 14:58:55 2022] PUT /identity/v3/projects/2822fc0546b94a28a0dbb37aee02f124/users/8c5ccfb591d14333b986fd0b9a3cf3be/roles/5d1feef0e13d4874928bd86258c4e3b2 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:58:55.788210 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b808e5be-c343-42fe-8873-72efd7890e41 None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:55.788552 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b808e5be-c343-42fe-8873-72efd7890e41 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:55.789309 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b808e5be-c343-42fe-8873-72efd7890e41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:55.887144 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-b808e5be-c343-42fe-8873-72efd7890e41 None None] MFA Rules not processed for user `8c5ccfb591d14333b986fd0b9a3cf3be`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:58:55.892428 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-b808e5be-c343-42fe-8873-72efd7890e41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:58:56.099214 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 913/1827] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 14:58:55 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 313 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:58:56.112286 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a60a459-4f5d-43f9-a379-1df12a5bbf2d admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:58:56.114495 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-1a60a459-4f5d-43f9-a379-1df12a5bbf2d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:58:56.347756 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a60a459-4f5d-43f9-a379-1df12a5bbf2d admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:58:56.370468 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8589a253-d2bd-4eab-9619-b739d04e23b3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3b81739e34e4e648f88b6d73fcb5e46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:58:56.371111 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8589a253-d2bd-4eab-9619-b739d04e23b3 service cinder] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:56.374808 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8589a253-d2bd-4eab-9619-b739d04e23b3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:56.375134 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8589a253-d2bd-4eab-9619-b739d04e23b3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:56.414310 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8589a253-d2bd-4eab-9619-b739d04e23b3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:58:56.415930 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8589a253-d2bd-4eab-9619-b739d04e23b3 service cinder] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:58:56.456679 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 915/1828] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 14:58:56 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 14:58:56.575796 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b808e5be-c343-42fe-8873-72efd7890e41 None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:58:56.577465 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-b808e5be-c343-42fe-8873-72efd7890e41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:58:56.875851 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b808e5be-c343-42fe-8873-72efd7890e41 None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:58:56.902061 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95f586f-1c0f-45d2-94bf-86164dfae003 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:58:56.906413 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95f586f-1c0f-45d2-94bf-86164dfae003 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:56.906692 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95f586f-1c0f-45d2-94bf-86164dfae003 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:56.906976 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95f586f-1c0f-45d2-94bf-86164dfae003 admin admin] PATH_INFO: `/v3/projects/7854e13fd33c40f19590afb99239f6eb` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:56.915942 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95f586f-1c0f-45d2-94bf-86164dfae003 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7854e13fd33c40f19590afb99239f6eb)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:58:56.918894 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95f586f-1c0f-45d2-94bf-86164dfae003 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:58:57.047153 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-e95f586f-1c0f-45d2-94bf-86164dfae003 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7854e13fd33c40f19590afb99239f6eb', 'request_id': 'req-e95f586f-1c0f-45d2-94bf-86164dfae003'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:58:57.059396 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-e95f586f-1c0f-45d2-94bf-86164dfae003 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7854e13fd33c40f19590afb99239f6eb', 'request_id': 'req-e95f586f-1c0f-45d2-94bf-86164dfae003'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:58:57.077374 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-e95f586f-1c0f-45d2-94bf-86164dfae003 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7854e13fd33c40f19590afb99239f6eb', 'request_id': 'req-e95f586f-1c0f-45d2-94bf-86164dfae003'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:58:57.083881 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cd0968-c88b-4637-9ffb-c306ae0be363 None None] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:57.084923 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cd0968-c88b-4637-9ffb-c306ae0be363 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:57.085052 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cd0968-c88b-4637-9ffb-c306ae0be363 None None] PATH_INFO: `/` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:57.086971 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:58:57.087740 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 916/1829] 10.222.0.33 () {54 vars in 921 bytes} [Thu Sep 1 14:58:57 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 14:58:57.100834 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-e95f586f-1c0f-45d2-94bf-86164dfae003 admin admin] The token cache is being invalidate because project 7854e13fd33c40f19590afb99239f6eb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81341) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:58:57.101449 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-e95f586f-1c0f-45d2-94bf-86164dfae003 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:58:57.102961 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-e95f586f-1c0f-45d2-94bf-86164dfae003 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:58:57.105435 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 914/1830] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Sep 1 14:58:56 2022] DELETE /identity/v3/projects/7854e13fd33c40f19590afb99239f6eb => generated 0 bytes in 530 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:58:57.109936 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f04d862-586e-4fef-b6f8-fbeb1d5620f6 None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:57.110211 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f04d862-586e-4fef-b6f8-fbeb1d5620f6 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:57.110696 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f04d862-586e-4fef-b6f8-fbeb1d5620f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:57.129061 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019c5506-565c-4f37-980f-42127e765fe8 None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:57.129342 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019c5506-565c-4f37-980f-42127e765fe8 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:57.129620 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019c5506-565c-4f37-980f-42127e765fe8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:57.142165 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-8f04d862-586e-4fef-b6f8-fbeb1d5620f6 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:58:57.144972 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-8f04d862-586e-4fef-b6f8-fbeb1d5620f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:58:57.231276 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-019c5506-565c-4f37-980f-42127e765fe8 None None] MFA Rules not processed for user `4858bb8621854ec8aba836e7d9050498`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:58:57.235933 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-019c5506-565c-4f37-980f-42127e765fe8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:58:57.380005 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 917/1831] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 14:58:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 270 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:58:57.387144 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd2015d-d168-4a9c-95bd-63d6274db997 None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:57.387473 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd2015d-d168-4a9c-95bd-63d6274db997 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:57.387681 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd2015d-d168-4a9c-95bd-63d6274db997 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:57.437077 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-2fd2015d-d168-4a9c-95bd-63d6274db997 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:58:57.439088 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-2fd2015d-d168-4a9c-95bd-63d6274db997 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:58:57.458584 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 918/1832] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 14:58:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 14:58:57.463056 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd2015d-d168-4a9c-95bd-63d6274db997 None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:58:57.477578 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd2015d-d168-4a9c-95bd-63d6274db997 None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:58:57.487998 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8674b1e3-d487-4c13-b4f6-8b21d2121800 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:58:57.488892 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8674b1e3-d487-4c13-b4f6-8b21d2121800 admin admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:57.489394 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8674b1e3-d487-4c13-b4f6-8b21d2121800 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:57.489970 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8674b1e3-d487-4c13-b4f6-8b21d2121800 admin admin] PATH_INFO: `/v3/domains` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:57.491806 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8674b1e3-d487-4c13-b4f6-8b21d2121800 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:58:57.492428 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8674b1e3-d487-4c13-b4f6-8b21d2121800 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:58:57.495162 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8674b1e3-d487-4c13-b4f6-8b21d2121800 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:58:57.513297 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 915/1833] 10.222.0.33 () {58 vars in 1018 bytes} [Thu Sep 1 14:58:57 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 385 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:58:57.523788 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 919/1834] 10.222.0.33 () {58 vars in 1153 bytes} [Thu Sep 1 14:58:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 14:58:57.524975 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019c5506-565c-4f37-980f-42127e765fe8 None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:58:57.569992 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019c5506-565c-4f37-980f-42127e765fe8 None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:58:57.585470 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba9121e-05b5-4bc7-8aa0-b28e5853702c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:58:57.586895 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba9121e-05b5-4bc7-8aa0-b28e5853702c admin admin] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:57.587986 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba9121e-05b5-4bc7-8aa0-b28e5853702c admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:57.590486 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba9121e-05b5-4bc7-8aa0-b28e5853702c admin admin] PATH_INFO: `/v3/projects` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:57.593052 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba9121e-05b5-4bc7-8aa0-b28e5853702c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:58:57.597057 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba9121e-05b5-4bc7-8aa0-b28e5853702c admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:58:57.674150 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 916/1835] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Sep 1 14:58:57 2022] POST /identity/v3/projects => generated 377 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:58:57.679011 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8674b1e3-d487-4c13-b4f6-8b21d2121800 admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:58:57.700239 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8674b1e3-d487-4c13-b4f6-8b21d2121800 admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:58:57.826592 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a1b553-c431-4e0f-bbc9-96808567cc85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:58:57.827223 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a1b553-c431-4e0f-bbc9-96808567cc85 admin admin] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:57.827617 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a1b553-c431-4e0f-bbc9-96808567cc85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:57.827930 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a1b553-c431-4e0f-bbc9-96808567cc85 admin admin] PATH_INFO: `/v3/users` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:57.828556 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a1b553-c431-4e0f-bbc9-96808567cc85 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:58:57.830733 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a1b553-c431-4e0f-bbc9-96808567cc85 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:58:57.865131 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 920/1836] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Sep 1 14:58:57 2022] POST /identity/v3/users => generated 289 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:58:57.875608 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba9121e-05b5-4bc7-8aa0-b28e5853702c admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:58:57.890718 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba9121e-05b5-4bc7-8aa0-b28e5853702c admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:58:57.897331 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d3f1d3-fe70-42fd-ae4d-036c4619fecc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:58:57.898343 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d3f1d3-fe70-42fd-ae4d-036c4619fecc admin admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:57.898850 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d3f1d3-fe70-42fd-ae4d-036c4619fecc admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:57.899342 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d3f1d3-fe70-42fd-ae4d-036c4619fecc admin admin] PATH_INFO: `/v3/roles` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:57.900218 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d3f1d3-fe70-42fd-ae4d-036c4619fecc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:58:57.902051 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d3f1d3-fe70-42fd-ae4d-036c4619fecc admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:58:57.924745 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 917/1837] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 1 14:58:57 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:58:57.925710 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a1b553-c431-4e0f-bbc9-96808567cc85 admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:58:57.940990 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a1b553-c431-4e0f-bbc9-96808567cc85 admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:58:57.949382 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66d9cd2-b7a3-4bba-83cd-0be4cdf7fcbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:58:57.950420 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66d9cd2-b7a3-4bba-83cd-0be4cdf7fcbe admin admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:57.950916 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66d9cd2-b7a3-4bba-83cd-0be4cdf7fcbe admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:57.951405 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66d9cd2-b7a3-4bba-83cd-0be4cdf7fcbe admin admin] PATH_INFO: `/v3/roles` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:57.952294 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66d9cd2-b7a3-4bba-83cd-0be4cdf7fcbe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:58:57.954577 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66d9cd2-b7a3-4bba-83cd-0be4cdf7fcbe admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:58:57.963027 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:58:57.975527 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 921/1838] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 1 14:58:57 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 14:58:57.976854 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d3f1d3-fe70-42fd-ae4d-036c4619fecc admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:58:57.990023 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d3f1d3-fe70-42fd-ae4d-036c4619fecc admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:58:57.997638 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c20643-44b3-4edb-a48a-7b50c145017b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:58:57.998817 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c20643-44b3-4edb-a48a-7b50c145017b admin admin] REQUEST_METHOD: `PUT` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:57.999369 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c20643-44b3-4edb-a48a-7b50c145017b admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:57.999841 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c20643-44b3-4edb-a48a-7b50c145017b admin admin] PATH_INFO: `/v3/projects/f298a11bbc664dbe97b07c7f1a84d975/users/16e9b302197f4d75b07c42db666c71a8/roles/5d1feef0e13d4874928bd86258c4e3b2` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:58.018079 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c20643-44b3-4edb-a48a-7b50c145017b admin admin] RBAC: Authorizing `identity:create_grant(project_id=f298a11bbc664dbe97b07c7f1a84d975, user_id=16e9b302197f4d75b07c42db666c71a8, role_id=5d1feef0e13d4874928bd86258c4e3b2)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:58:58.021939 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c20643-44b3-4edb-a48a-7b50c145017b admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:58:58.048316 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 918/1839] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 1 14:58:57 2022] PUT /identity/v3/projects/f298a11bbc664dbe97b07c7f1a84d975/users/16e9b302197f4d75b07c42db666c71a8/roles/5d1feef0e13d4874928bd86258c4e3b2 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:58:58.050758 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cdbed8-d04e-4859-89e0-f4d284935d75 None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:58.051131 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cdbed8-d04e-4859-89e0-f4d284935d75 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:58.051533 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cdbed8-d04e-4859-89e0-f4d284935d75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:58.099952 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-35cdbed8-d04e-4859-89e0-f4d284935d75 None None] MFA Rules not processed for user `16e9b302197f4d75b07c42db666c71a8`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:58:58.102044 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-35cdbed8-d04e-4859-89e0-f4d284935d75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:58:58.221274 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 922/1840] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 14:58:58 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:58:58.225605 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c20643-44b3-4edb-a48a-7b50c145017b admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:58:58.293518 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c20643-44b3-4edb-a48a-7b50c145017b admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:58:58.301100 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1968011a-fa58-40a6-a4fe-a35bc56ac894 None None] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:58.301100 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1968011a-fa58-40a6-a4fe-a35bc56ac894 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:58.301100 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1968011a-fa58-40a6-a4fe-a35bc56ac894 None None] PATH_INFO: `/` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:58.302268 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8660a0c5-1508-4621-8f7b-a6809bdd864b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:58:58.303007 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8660a0c5-1508-4621-8f7b-a6809bdd864b admin admin] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:58.303393 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8660a0c5-1508-4621-8f7b-a6809bdd864b admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:58.304063 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8660a0c5-1508-4621-8f7b-a6809bdd864b admin admin] PATH_INFO: `/v3/projects` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:58.304792 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8660a0c5-1508-4621-8f7b-a6809bdd864b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:58:58.305761 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 923/1841] 10.222.0.33 () {54 vars in 921 bytes} [Thu Sep 1 14:58:58 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 14:58:58.307679 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8660a0c5-1508-4621-8f7b-a6809bdd864b admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:58:58.330130 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c7a72d-02db-48cc-bf12-472117546819 None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:58.330613 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c7a72d-02db-48cc-bf12-472117546819 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:58.330952 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c7a72d-02db-48cc-bf12-472117546819 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:58.342654 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 919/1842] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Sep 1 14:58:58 2022] POST /identity/v3/projects => generated 377 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 14:58:58.346185 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8660a0c5-1508-4621-8f7b-a6809bdd864b admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:58:58.357630 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-b0c7a72d-02db-48cc-bf12-472117546819 None None] MFA Rules not processed for user `4858bb8621854ec8aba836e7d9050498`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:58:58.358845 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-b0c7a72d-02db-48cc-bf12-472117546819 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:58:58.400307 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:58:58.491934 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 924/1843] 10.222.0.33 () {58 vars in 1018 bytes} [Thu Sep 1 14:58:58 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:58:58.492969 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c7a72d-02db-48cc-bf12-472117546819 None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:58:58.496667 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-b0c7a72d-02db-48cc-bf12-472117546819 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:58:58.515365 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c7a72d-02db-48cc-bf12-472117546819 None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:58:58.543144 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8660a0c5-1508-4621-8f7b-a6809bdd864b admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:58:58.543619 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8615a8-5d95-4d0a-af13-c7414e3ced04 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4858bb8621854ec8aba836e7d9050498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:58:58.544460 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8615a8-5d95-4d0a-af13-c7414e3ced04 service nova] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:58.544816 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8615a8-5d95-4d0a-af13-c7414e3ced04 service nova] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:58.547166 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8615a8-5d95-4d0a-af13-c7414e3ced04 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:58.552597 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8c31e4-c747-4b5e-9c44-49e6e1fd20cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:58:58.553366 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8c31e4-c747-4b5e-9c44-49e6e1fd20cf admin admin] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:58.553585 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8c31e4-c747-4b5e-9c44-49e6e1fd20cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:58.554128 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8c31e4-c747-4b5e-9c44-49e6e1fd20cf admin admin] PATH_INFO: `/v3/users` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:58.557875 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8c31e4-c747-4b5e-9c44-49e6e1fd20cf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:58:58.562543 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8c31e4-c747-4b5e-9c44-49e6e1fd20cf admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:58:58.570279 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8615a8-5d95-4d0a-af13-c7414e3ced04 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:58:58.573179 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8615a8-5d95-4d0a-af13-c7414e3ced04 service nova] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:58:58.606889 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 925/1844] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 14:58:58 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:58:58.628150 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 920/1845] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Sep 1 14:58:58 2022] POST /identity/v3/users => generated 289 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 14:58:58.628609 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8615a8-5d95-4d0a-af13-c7414e3ced04 service nova] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:58:58.638836 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8615a8-5d95-4d0a-af13-c7414e3ced04 service nova] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:58:58.648580 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aaf7c38-786b-4baf-ad5d-d47ee9ab4c08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:58:58.649613 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aaf7c38-786b-4baf-ad5d-d47ee9ab4c08 admin admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:58.649991 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aaf7c38-786b-4baf-ad5d-d47ee9ab4c08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:58.650386 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aaf7c38-786b-4baf-ad5d-d47ee9ab4c08 admin admin] PATH_INFO: `/v3/roles` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:58.651387 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aaf7c38-786b-4baf-ad5d-d47ee9ab4c08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:58:58.653408 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aaf7c38-786b-4baf-ad5d-d47ee9ab4c08 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:58:58.684991 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 926/1846] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 1 14:58:58 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:58:58.689633 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8c31e4-c747-4b5e-9c44-49e6e1fd20cf admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:58:58.715247 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8c31e4-c747-4b5e-9c44-49e6e1fd20cf admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:58:58.746544 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ba7d08-849c-44f0-bbbc-17566c3bf6d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:58:58.758916 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ba7d08-849c-44f0-bbbc-17566c3bf6d9 admin admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:58.759246 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ba7d08-849c-44f0-bbbc-17566c3bf6d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:58.759561 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ba7d08-849c-44f0-bbbc-17566c3bf6d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:58.760246 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ba7d08-849c-44f0-bbbc-17566c3bf6d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:58:58.763495 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ba7d08-849c-44f0-bbbc-17566c3bf6d9 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:58:58.773491 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aaf7c38-786b-4baf-ad5d-d47ee9ab4c08 admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:58:58.791474 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aaf7c38-786b-4baf-ad5d-d47ee9ab4c08 admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:58:58.801332 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 921/1847] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 1 14:58:58 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:58:58.806978 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ba7d08-849c-44f0-bbbc-17566c3bf6d9 admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:58:58.808850 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b416235a-b031-4ea3-9d5c-cac461e3e7ec service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4858bb8621854ec8aba836e7d9050498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:58:58.809828 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b416235a-b031-4ea3-9d5c-cac461e3e7ec service nova] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:58.810230 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b416235a-b031-4ea3-9d5c-cac461e3e7ec service nova] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:58.811922 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b416235a-b031-4ea3-9d5c-cac461e3e7ec service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:58.824408 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b416235a-b031-4ea3-9d5c-cac461e3e7ec service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:58:58.833135 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ba7d08-849c-44f0-bbbc-17566c3bf6d9 admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:58:58.837758 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b416235a-b031-4ea3-9d5c-cac461e3e7ec service nova] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:58:58.849133 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03594f30-3232-4c6c-9fda-ab409fe9b9e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:58:58.849859 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03594f30-3232-4c6c-9fda-ab409fe9b9e2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:58.855644 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03594f30-3232-4c6c-9fda-ab409fe9b9e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:58.855644 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03594f30-3232-4c6c-9fda-ab409fe9b9e2 admin admin] PATH_INFO: `/v3/projects/aae5184a9e4c46e7b71a3c9c5516422b/users/6f49ce0282bd41a3b85706c584453362/roles/5d1feef0e13d4874928bd86258c4e3b2` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:58.878195 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 927/1848] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 14:58:58 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 14:58:58.891486 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03594f30-3232-4c6c-9fda-ab409fe9b9e2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aae5184a9e4c46e7b71a3c9c5516422b, user_id=6f49ce0282bd41a3b85706c584453362, role_id=5d1feef0e13d4874928bd86258c4e3b2)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:58:58.896835 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03594f30-3232-4c6c-9fda-ab409fe9b9e2 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:58:58.934127 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 922/1849] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 1 14:58:58 2022] PUT /identity/v3/projects/aae5184a9e4c46e7b71a3c9c5516422b/users/6f49ce0282bd41a3b85706c584453362/roles/5d1feef0e13d4874928bd86258c4e3b2 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:58:58.951586 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79661d8c-9368-4812-8de0-496c3c0c4a7f None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:58.952126 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79661d8c-9368-4812-8de0-496c3c0c4a7f None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:58.952631 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79661d8c-9368-4812-8de0-496c3c0c4a7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:58.989853 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-79661d8c-9368-4812-8de0-496c3c0c4a7f None None] MFA Rules not processed for user `6f49ce0282bd41a3b85706c584453362`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:58:58.991620 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-79661d8c-9368-4812-8de0-496c3c0c4a7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:58:59.044130 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:58:59.111781 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 928/1850] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 14:58:58 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:58:59.117346 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03594f30-3232-4c6c-9fda-ab409fe9b9e2 admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:58:59.192786 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03594f30-3232-4c6c-9fda-ab409fe9b9e2 admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:58:59.200610 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be422dc-d0ac-46c2-b807-60d33f3f0729 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:58:59.201392 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be422dc-d0ac-46c2-b807-60d33f3f0729 admin admin] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:59.202106 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be422dc-d0ac-46c2-b807-60d33f3f0729 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:59.202106 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be422dc-d0ac-46c2-b807-60d33f3f0729 admin admin] PATH_INFO: `/v3/projects` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:59.202880 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be422dc-d0ac-46c2-b807-60d33f3f0729 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:58:59.204993 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be422dc-d0ac-46c2-b807-60d33f3f0729 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:58:59.250631 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 923/1851] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Sep 1 14:58:59 2022] POST /identity/v3/projects => generated 379 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 14:58:59.253786 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79661d8c-9368-4812-8de0-496c3c0c4a7f None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:58:59.275683 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79661d8c-9368-4812-8de0-496c3c0c4a7f None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:58:59.378081 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3a01c1-42f0-4d9e-b663-920004c39cb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:58:59.379434 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3a01c1-42f0-4d9e-b663-920004c39cb4 admin admin] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:59.379818 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3a01c1-42f0-4d9e-b663-920004c39cb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:59.380191 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3a01c1-42f0-4d9e-b663-920004c39cb4 admin admin] PATH_INFO: `/v3/users` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:59.380946 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3a01c1-42f0-4d9e-b663-920004c39cb4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:58:59.383174 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3a01c1-42f0-4d9e-b663-920004c39cb4 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:58:59.430279 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 929/1852] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Sep 1 14:58:59 2022] POST /identity/v3/users => generated 296 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:58:59.430418 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be422dc-d0ac-46c2-b807-60d33f3f0729 admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:58:59.452713 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be422dc-d0ac-46c2-b807-60d33f3f0729 admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:58:59.458912 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68930a94-6e49-4bd6-bf99-5063d9ead433 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:58:59.459530 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68930a94-6e49-4bd6-bf99-5063d9ead433 admin admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:59.459810 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68930a94-6e49-4bd6-bf99-5063d9ead433 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:59.460079 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68930a94-6e49-4bd6-bf99-5063d9ead433 admin admin] PATH_INFO: `/v3/roles` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:59.460725 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68930a94-6e49-4bd6-bf99-5063d9ead433 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:58:59.462542 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68930a94-6e49-4bd6-bf99-5063d9ead433 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:58:59.476487 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 924/1853] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 1 14:58:59 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:58:59.482313 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3a01c1-42f0-4d9e-b663-920004c39cb4 admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:58:59.501933 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3a01c1-42f0-4d9e-b663-920004c39cb4 admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:58:59.512623 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3054b81-bc86-462d-bba6-0e47b07262d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:58:59.513944 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3054b81-bc86-462d-bba6-0e47b07262d0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:59.514517 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3054b81-bc86-462d-bba6-0e47b07262d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:59.515033 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3054b81-bc86-462d-bba6-0e47b07262d0 admin admin] PATH_INFO: `/v3/projects/4896268c46ee41549356477e4932e02d/users/3d6583a2d5694b398f23efbefdc1098d/roles/2a2adb769e55476b82e5014897e0a9c0` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:59.537222 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3054b81-bc86-462d-bba6-0e47b07262d0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4896268c46ee41549356477e4932e02d, user_id=3d6583a2d5694b398f23efbefdc1098d, role_id=2a2adb769e55476b82e5014897e0a9c0)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:58:59.542069 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3054b81-bc86-462d-bba6-0e47b07262d0 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:58:59.556769 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 930/1854] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 1 14:58:59 2022] PUT /identity/v3/projects/4896268c46ee41549356477e4932e02d/users/3d6583a2d5694b398f23efbefdc1098d/roles/2a2adb769e55476b82e5014897e0a9c0 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:58:59.561110 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68930a94-6e49-4bd6-bf99-5063d9ead433 admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:58:59.572851 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68930a94-6e49-4bd6-bf99-5063d9ead433 admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:58:59.644099 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:58:59.679956 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf985f2-62cc-4b52-970a-36186e2b0c12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:58:59.681076 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf985f2-62cc-4b52-970a-36186e2b0c12 admin admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:59.682123 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf985f2-62cc-4b52-970a-36186e2b0c12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:59.682653 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf985f2-62cc-4b52-970a-36186e2b0c12 admin admin] PATH_INFO: `/v3/roles` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:59.683727 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf985f2-62cc-4b52-970a-36186e2b0c12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:58:59.686571 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf985f2-62cc-4b52-970a-36186e2b0c12 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:58:59.707389 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3054b81-bc86-462d-bba6-0e47b07262d0 admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:58:59.712421 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 925/1855] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 1 14:58:59 2022] GET /identity/v3/roles => generated 1420 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:58:59.729265 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3054b81-bc86-462d-bba6-0e47b07262d0 admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:58:59.758129 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012b53d6-c193-4492-b23e-d961d6f6ed16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:58:59.759028 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012b53d6-c193-4492-b23e-d961d6f6ed16 admin admin] REQUEST_METHOD: `PUT` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:59.759458 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012b53d6-c193-4492-b23e-d961d6f6ed16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:59.759896 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012b53d6-c193-4492-b23e-d961d6f6ed16 admin admin] PATH_INFO: `/v3/projects/4896268c46ee41549356477e4932e02d/users/3d6583a2d5694b398f23efbefdc1098d/roles/2a2adb769e55476b82e5014897e0a9c0` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:59.764979 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012b53d6-c193-4492-b23e-d961d6f6ed16 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4896268c46ee41549356477e4932e02d, user_id=3d6583a2d5694b398f23efbefdc1098d, role_id=2a2adb769e55476b82e5014897e0a9c0)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:58:59.769729 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012b53d6-c193-4492-b23e-d961d6f6ed16 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:58:59.807641 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 931/1856] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 1 14:58:59 2022] PUT /identity/v3/projects/4896268c46ee41549356477e4932e02d/users/3d6583a2d5694b398f23efbefdc1098d/roles/2a2adb769e55476b82e5014897e0a9c0 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:58:59.819438 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a209783-8b0a-4c81-becb-2c8e828d61c8 None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:58:59.819725 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a209783-8b0a-4c81-becb-2c8e828d61c8 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:58:59.820032 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a209783-8b0a-4c81-becb-2c8e828d61c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:58:59.898694 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-8a209783-8b0a-4c81-becb-2c8e828d61c8 None None] MFA Rules not processed for user `3d6583a2d5694b398f23efbefdc1098d`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:58:59.905866 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-8a209783-8b0a-4c81-becb-2c8e828d61c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:59:00.015959 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 926/1857] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 14:58:59 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:59:00.030176 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012b53d6-c193-4492-b23e-d961d6f6ed16 admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:59:00.032725 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-012b53d6-c193-4492-b23e-d961d6f6ed16 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:59:00.106958 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012b53d6-c193-4492-b23e-d961d6f6ed16 admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:59:00.111432 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc9dc61-e40d-41aa-bd61-47a8a550d142 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3b81739e34e4e648f88b6d73fcb5e46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:59:00.112161 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc9dc61-e40d-41aa-bd61-47a8a550d142 service cinder] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:00.113892 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc9dc61-e40d-41aa-bd61-47a8a550d142 service cinder] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:00.114216 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc9dc61-e40d-41aa-bd61-47a8a550d142 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:00.127401 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc9dc61-e40d-41aa-bd61-47a8a550d142 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:59:00.129430 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc9dc61-e40d-41aa-bd61-47a8a550d142 service cinder] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:59:00.144998 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 932/1858] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 14:59:00 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:59:00.211628 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a209783-8b0a-4c81-becb-2c8e828d61c8 None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:59:00.223840 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a209783-8b0a-4c81-becb-2c8e828d61c8 None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:59:00.229502 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3887e8c4-c0bf-4d9a-9fb5-097faf3f0fc2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3b81739e34e4e648f88b6d73fcb5e46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:59:00.230171 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3887e8c4-c0bf-4d9a-9fb5-097faf3f0fc2 service cinder] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:00.230528 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3887e8c4-c0bf-4d9a-9fb5-097faf3f0fc2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:00.231039 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3887e8c4-c0bf-4d9a-9fb5-097faf3f0fc2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:00.358978 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3887e8c4-c0bf-4d9a-9fb5-097faf3f0fc2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:59:00.361580 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3887e8c4-c0bf-4d9a-9fb5-097faf3f0fc2 service cinder] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:59:00.414187 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 927/1859] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 14:59:00 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:59:00.851854 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc9dc61-e40d-41aa-bd61-47a8a550d142 service cinder] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:59:00.853370 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-8bc9dc61-e40d-41aa-bd61-47a8a550d142 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:59:00.933548 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:59:01.003424 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc9dc61-e40d-41aa-bd61-47a8a550d142 service cinder] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:59:01.025256 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54431f63-b078-4628-8884-06072baf6c21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:59:01.030053 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54431f63-b078-4628-8884-06072baf6c21 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:01.030910 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54431f63-b078-4628-8884-06072baf6c21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:01.031701 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54431f63-b078-4628-8884-06072baf6c21 admin admin] PATH_INFO: `/v3/users/8c5ccfb591d14333b986fd0b9a3cf3be` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:01.035163 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54431f63-b078-4628-8884-06072baf6c21 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8c5ccfb591d14333b986fd0b9a3cf3be)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:59:01.040529 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54431f63-b078-4628-8884-06072baf6c21 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:59:01.127059 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-54431f63-b078-4628-8884-06072baf6c21 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8c5ccfb591d14333b986fd0b9a3cf3be'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:01.142223 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-54431f63-b078-4628-8884-06072baf6c21 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8c5ccfb591d14333b986fd0b9a3cf3be'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:01.146341 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-54431f63-b078-4628-8884-06072baf6c21 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8c5ccfb591d14333b986fd0b9a3cf3be'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:01.161142 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-54431f63-b078-4628-8884-06072baf6c21 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8c5ccfb591d14333b986fd0b9a3cf3be'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:01.162038 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-54431f63-b078-4628-8884-06072baf6c21 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8c5ccfb591d14333b986fd0b9a3cf3be'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:01.173902 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 933/1860] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Sep 1 14:59:00 2022] DELETE /identity/v3/users/8c5ccfb591d14333b986fd0b9a3cf3be => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:59:01.179175 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fc1054-c855-4c20-8ff5-25428b23a142 None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:01.179672 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fc1054-c855-4c20-8ff5-25428b23a142 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:01.179766 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fc1054-c855-4c20-8ff5-25428b23a142 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:01.199879 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-47fc1054-c855-4c20-8ff5-25428b23a142 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:59:01.201071 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-47fc1054-c855-4c20-8ff5-25428b23a142 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:59:01.257584 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 928/1861] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 14:59:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:59:01.267388 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54431f63-b078-4628-8884-06072baf6c21 admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:59:01.268792 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-54431f63-b078-4628-8884-06072baf6c21 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:59:01.326019 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54431f63-b078-4628-8884-06072baf6c21 admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:59:01.329647 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d87c254-2019-4a5c-aab9-31552c77e905 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2790149639aa418bbc4e4982e93eda44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:59:01.330276 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d87c254-2019-4a5c-aab9-31552c77e905 service neutron] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:01.330562 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d87c254-2019-4a5c-aab9-31552c77e905 service neutron] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:01.330835 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d87c254-2019-4a5c-aab9-31552c77e905 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:01.339690 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d87c254-2019-4a5c-aab9-31552c77e905 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:59:01.341143 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d87c254-2019-4a5c-aab9-31552c77e905 service neutron] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:59:01.352130 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 934/1862] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 14:59:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:59:01.629172 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fc1054-c855-4c20-8ff5-25428b23a142 None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:59:01.630762 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-47fc1054-c855-4c20-8ff5-25428b23a142 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:59:01.639627 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:59:01.644368 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fc1054-c855-4c20-8ff5-25428b23a142 None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:59:01.649338 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff606a7-f44b-4c02-bb8e-a811796aec72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:59:01.649869 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff606a7-f44b-4c02-bb8e-a811796aec72 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:01.650344 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff606a7-f44b-4c02-bb8e-a811796aec72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:01.650627 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff606a7-f44b-4c02-bb8e-a811796aec72 admin admin] PATH_INFO: `/v3/projects/2822fc0546b94a28a0dbb37aee02f124` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:01.655314 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff606a7-f44b-4c02-bb8e-a811796aec72 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2822fc0546b94a28a0dbb37aee02f124)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:59:01.655314 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff606a7-f44b-4c02-bb8e-a811796aec72 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:59:01.704892 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-1ff606a7-f44b-4c02-bb8e-a811796aec72 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2822fc0546b94a28a0dbb37aee02f124', 'request_id': 'req-1ff606a7-f44b-4c02-bb8e-a811796aec72'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:01.711201 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-1ff606a7-f44b-4c02-bb8e-a811796aec72 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2822fc0546b94a28a0dbb37aee02f124', 'request_id': 'req-1ff606a7-f44b-4c02-bb8e-a811796aec72'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:01.718937 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-1ff606a7-f44b-4c02-bb8e-a811796aec72 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2822fc0546b94a28a0dbb37aee02f124', 'request_id': 'req-1ff606a7-f44b-4c02-bb8e-a811796aec72'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:01.731247 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-1ff606a7-f44b-4c02-bb8e-a811796aec72 admin admin] The token cache is being invalidate because project 2822fc0546b94a28a0dbb37aee02f124 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81341) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:59:01.731535 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-1ff606a7-f44b-4c02-bb8e-a811796aec72 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:01.732240 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-1ff606a7-f44b-4c02-bb8e-a811796aec72 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:01.733725 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 929/1863] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Sep 1 14:59:01 2022] DELETE /identity/v3/projects/2822fc0546b94a28a0dbb37aee02f124 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:59:01.739325 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163da104-69d5-482b-a458-9cb5e0f3047d None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:01.739757 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163da104-69d5-482b-a458-9cb5e0f3047d None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:01.739937 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163da104-69d5-482b-a458-9cb5e0f3047d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:01.761434 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-163da104-69d5-482b-a458-9cb5e0f3047d None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:59:01.762712 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-163da104-69d5-482b-a458-9cb5e0f3047d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:59:01.819125 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:59:01.839774 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 935/1864] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 14:59:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 14:59:01.847474 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2af788e-24f5-41fe-a37f-40ce0a5b8999 None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:01.847784 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2af788e-24f5-41fe-a37f-40ce0a5b8999 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:01.848056 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2af788e-24f5-41fe-a37f-40ce0a5b8999 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:01.874113 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-d2af788e-24f5-41fe-a37f-40ce0a5b8999 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:59:01.875672 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-d2af788e-24f5-41fe-a37f-40ce0a5b8999 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:59:01.884307 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 930/1865] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 14:59:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:59:01.888537 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163da104-69d5-482b-a458-9cb5e0f3047d None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:59:01.898199 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163da104-69d5-482b-a458-9cb5e0f3047d None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:59:01.902862 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06170e1-4945-44e9-9ad6-be8192672072 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:59:01.903467 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06170e1-4945-44e9-9ad6-be8192672072 admin admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:01.903748 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06170e1-4945-44e9-9ad6-be8192672072 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:01.904020 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06170e1-4945-44e9-9ad6-be8192672072 admin admin] PATH_INFO: `/v3/domains` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:01.904674 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06170e1-4945-44e9-9ad6-be8192672072 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:59:01.904977 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06170e1-4945-44e9-9ad6-be8192672072 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:59:01.906631 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06170e1-4945-44e9-9ad6-be8192672072 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:59:01.918621 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 936/1866] 10.222.0.33 () {58 vars in 1153 bytes} [Thu Sep 1 14:59:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:59:01.923240 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2af788e-24f5-41fe-a37f-40ce0a5b8999 None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:59:01.933433 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2af788e-24f5-41fe-a37f-40ce0a5b8999 None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:59:01.938041 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6008872-b0a7-4820-82d5-b7a69f270f0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:59:01.938652 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6008872-b0a7-4820-82d5-b7a69f270f0d admin admin] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:01.938932 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6008872-b0a7-4820-82d5-b7a69f270f0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:01.939197 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6008872-b0a7-4820-82d5-b7a69f270f0d admin admin] PATH_INFO: `/v3/projects` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:01.939867 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6008872-b0a7-4820-82d5-b7a69f270f0d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:59:01.942405 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6008872-b0a7-4820-82d5-b7a69f270f0d admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:59:01.973515 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 931/1867] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Sep 1 14:59:01 2022] POST /identity/v3/projects => generated 385 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 14:59:01.974668 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06170e1-4945-44e9-9ad6-be8192672072 admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:59:01.985466 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06170e1-4945-44e9-9ad6-be8192672072 admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:59:02.041248 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2b0d92-e37d-4b18-9132-b4ce7ad9dad0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:59:02.041848 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2b0d92-e37d-4b18-9132-b4ce7ad9dad0 admin admin] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:02.042123 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2b0d92-e37d-4b18-9132-b4ce7ad9dad0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:02.042421 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2b0d92-e37d-4b18-9132-b4ce7ad9dad0 admin admin] PATH_INFO: `/v3/users` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:02.043034 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2b0d92-e37d-4b18-9132-b4ce7ad9dad0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:59:02.044937 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2b0d92-e37d-4b18-9132-b4ce7ad9dad0 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:59:02.071910 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 937/1868] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Sep 1 14:59:01 2022] POST /identity/v3/users => generated 293 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 14:59:02.075903 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6008872-b0a7-4820-82d5-b7a69f270f0d admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:59:02.087137 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6008872-b0a7-4820-82d5-b7a69f270f0d admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:59:02.092050 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f222813-d2e5-49bd-b484-34e766ed5942 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:59:02.092648 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f222813-d2e5-49bd-b484-34e766ed5942 admin admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:02.092923 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f222813-d2e5-49bd-b484-34e766ed5942 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:02.093192 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f222813-d2e5-49bd-b484-34e766ed5942 admin admin] PATH_INFO: `/v3/roles` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:02.093832 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f222813-d2e5-49bd-b484-34e766ed5942 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:59:02.095475 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f222813-d2e5-49bd-b484-34e766ed5942 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:59:02.106802 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 932/1869] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 1 14:59:02 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:59:02.111040 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2b0d92-e37d-4b18-9132-b4ce7ad9dad0 admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:59:02.121954 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2b0d92-e37d-4b18-9132-b4ce7ad9dad0 admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:59:02.126685 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec65791-0609-41fe-819f-1a4f5dd328d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:59:02.127305 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec65791-0609-41fe-819f-1a4f5dd328d7 admin admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:02.128182 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec65791-0609-41fe-819f-1a4f5dd328d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:02.128449 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec65791-0609-41fe-819f-1a4f5dd328d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:02.129756 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec65791-0609-41fe-819f-1a4f5dd328d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:59:02.131363 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec65791-0609-41fe-819f-1a4f5dd328d7 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:59:02.143652 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 938/1870] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 1 14:59:02 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 14:59:02.147843 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f222813-d2e5-49bd-b484-34e766ed5942 admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:59:02.158224 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f222813-d2e5-49bd-b484-34e766ed5942 admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:59:02.162973 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a8b33e-0cbd-4c38-b948-1c3472482458 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:59:02.163694 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a8b33e-0cbd-4c38-b948-1c3472482458 admin admin] REQUEST_METHOD: `PUT` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:02.164175 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a8b33e-0cbd-4c38-b948-1c3472482458 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:02.164418 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a8b33e-0cbd-4c38-b948-1c3472482458 admin admin] PATH_INFO: `/v3/projects/56203d292bbb4cb2bae5bc2f3a34d080/users/208e86a50cac462cb420ff0f08b9cbbc/roles/5d1feef0e13d4874928bd86258c4e3b2` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:02.180213 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a8b33e-0cbd-4c38-b948-1c3472482458 admin admin] RBAC: Authorizing `identity:create_grant(project_id=56203d292bbb4cb2bae5bc2f3a34d080, user_id=208e86a50cac462cb420ff0f08b9cbbc, role_id=5d1feef0e13d4874928bd86258c4e3b2)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:59:02.182984 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a8b33e-0cbd-4c38-b948-1c3472482458 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:59:02.199187 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 933/1871] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 1 14:59:02 2022] PUT /identity/v3/projects/56203d292bbb4cb2bae5bc2f3a34d080/users/208e86a50cac462cb420ff0f08b9cbbc/roles/5d1feef0e13d4874928bd86258c4e3b2 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:59:02.209778 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669f4f89-9826-4cd9-ba3f-920308e63775 None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:02.210061 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669f4f89-9826-4cd9-ba3f-920308e63775 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:02.210373 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669f4f89-9826-4cd9-ba3f-920308e63775 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:02.238460 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-669f4f89-9826-4cd9-ba3f-920308e63775 None None] MFA Rules not processed for user `208e86a50cac462cb420ff0f08b9cbbc`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:59:02.239688 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-669f4f89-9826-4cd9-ba3f-920308e63775 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:59:02.303279 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 939/1872] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 14:59:02 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:59:02.308136 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a8b33e-0cbd-4c38-b948-1c3472482458 admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:59:02.368492 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a8b33e-0cbd-4c38-b948-1c3472482458 admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:59:02.373090 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c296e4-6a17-4256-9c25-2494ea70dc2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:59:02.373686 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c296e4-6a17-4256-9c25-2494ea70dc2c admin admin] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:02.373964 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c296e4-6a17-4256-9c25-2494ea70dc2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:02.374256 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c296e4-6a17-4256-9c25-2494ea70dc2c admin admin] PATH_INFO: `/v3/projects` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:02.374896 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c296e4-6a17-4256-9c25-2494ea70dc2c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:59:02.376813 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c296e4-6a17-4256-9c25-2494ea70dc2c admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:59:02.402179 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 934/1873] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Sep 1 14:59:02 2022] POST /identity/v3/projects => generated 383 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:59:02.406211 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669f4f89-9826-4cd9-ba3f-920308e63775 None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:59:02.417381 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669f4f89-9826-4cd9-ba3f-920308e63775 None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:59:02.460127 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:59:02.473989 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94465a0c-8afa-42b1-9577-1de8919afdab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:59:02.474639 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94465a0c-8afa-42b1-9577-1de8919afdab admin admin] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:02.474916 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94465a0c-8afa-42b1-9577-1de8919afdab admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:02.475182 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94465a0c-8afa-42b1-9577-1de8919afdab admin admin] PATH_INFO: `/v3/users` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:02.475818 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94465a0c-8afa-42b1-9577-1de8919afdab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:59:02.477735 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94465a0c-8afa-42b1-9577-1de8919afdab admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:59:02.498469 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 940/1874] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Sep 1 14:59:02 2022] POST /identity/v3/users => generated 292 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 14:59:02.502411 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c296e4-6a17-4256-9c25-2494ea70dc2c admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:59:02.512987 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c296e4-6a17-4256-9c25-2494ea70dc2c admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:59:02.517587 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c57724-0a99-4128-8905-e1cf8b9cf82a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:59:02.518180 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c57724-0a99-4128-8905-e1cf8b9cf82a admin admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:02.518492 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c57724-0a99-4128-8905-e1cf8b9cf82a admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:02.518763 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c57724-0a99-4128-8905-e1cf8b9cf82a admin admin] PATH_INFO: `/v3/roles` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:02.519435 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c57724-0a99-4128-8905-e1cf8b9cf82a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:59:02.521049 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c57724-0a99-4128-8905-e1cf8b9cf82a admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:59:02.523112 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:59:02.533533 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 935/1875] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 1 14:59:02 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 14:59:02.537344 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94465a0c-8afa-42b1-9577-1de8919afdab admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:59:02.547031 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94465a0c-8afa-42b1-9577-1de8919afdab admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:59:02.552102 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364f7e29-d8fd-49e5-a907-4248e7910df1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:59:02.552685 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364f7e29-d8fd-49e5-a907-4248e7910df1 admin admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:02.552961 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364f7e29-d8fd-49e5-a907-4248e7910df1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:02.553227 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364f7e29-d8fd-49e5-a907-4248e7910df1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:02.554131 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364f7e29-d8fd-49e5-a907-4248e7910df1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:59:02.555731 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364f7e29-d8fd-49e5-a907-4248e7910df1 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:59:02.565308 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 941/1876] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 1 14:59:02 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:59:02.569232 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c57724-0a99-4128-8905-e1cf8b9cf82a admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:59:02.579093 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c57724-0a99-4128-8905-e1cf8b9cf82a admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:59:02.583919 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac0f036-9834-4a52-8c2f-e40f38642ac6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:59:02.584628 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac0f036-9834-4a52-8c2f-e40f38642ac6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:02.584896 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac0f036-9834-4a52-8c2f-e40f38642ac6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:02.585199 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac0f036-9834-4a52-8c2f-e40f38642ac6 admin admin] PATH_INFO: `/v3/projects/69f76f3d089e433e85311d7ceb2ee087/users/93ddb476fc534556a9bc37ddcc9a5c12/roles/5d1feef0e13d4874928bd86258c4e3b2` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:02.600335 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac0f036-9834-4a52-8c2f-e40f38642ac6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=69f76f3d089e433e85311d7ceb2ee087, user_id=93ddb476fc534556a9bc37ddcc9a5c12, role_id=5d1feef0e13d4874928bd86258c4e3b2)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:59:02.603073 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac0f036-9834-4a52-8c2f-e40f38642ac6 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:59:02.617694 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 936/1877] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 1 14:59:02 2022] PUT /identity/v3/projects/69f76f3d089e433e85311d7ceb2ee087/users/93ddb476fc534556a9bc37ddcc9a5c12/roles/5d1feef0e13d4874928bd86258c4e3b2 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:59:02.630342 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11487fda-b47b-42a6-8d0e-24dbc06f0551 None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:02.631473 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11487fda-b47b-42a6-8d0e-24dbc06f0551 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:02.632409 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11487fda-b47b-42a6-8d0e-24dbc06f0551 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:02.670137 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-11487fda-b47b-42a6-8d0e-24dbc06f0551 None None] MFA Rules not processed for user `93ddb476fc534556a9bc37ddcc9a5c12`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:59:02.671435 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-11487fda-b47b-42a6-8d0e-24dbc06f0551 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:59:02.735229 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 942/1878] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 14:59:02 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:59:02.739719 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac0f036-9834-4a52-8c2f-e40f38642ac6 admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:59:02.799892 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac0f036-9834-4a52-8c2f-e40f38642ac6 admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:59:02.804591 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccefac04-e0f4-443e-945f-a32af4d86505 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:59:02.805189 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccefac04-e0f4-443e-945f-a32af4d86505 admin admin] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:02.805468 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccefac04-e0f4-443e-945f-a32af4d86505 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:02.805739 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccefac04-e0f4-443e-945f-a32af4d86505 admin admin] PATH_INFO: `/v3/projects` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:02.806406 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccefac04-e0f4-443e-945f-a32af4d86505 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:59:02.808316 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccefac04-e0f4-443e-945f-a32af4d86505 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:59:02.837510 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 937/1879] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Sep 1 14:59:02 2022] POST /identity/v3/projects => generated 385 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:59:02.841514 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11487fda-b47b-42a6-8d0e-24dbc06f0551 None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:59:02.852624 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11487fda-b47b-42a6-8d0e-24dbc06f0551 None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:59:02.911406 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06af5490-7d41-4e2b-8993-d8559a4e66c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:59:02.912028 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06af5490-7d41-4e2b-8993-d8559a4e66c5 admin admin] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:02.912306 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06af5490-7d41-4e2b-8993-d8559a4e66c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:02.912573 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06af5490-7d41-4e2b-8993-d8559a4e66c5 admin admin] PATH_INFO: `/v3/users` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:02.913230 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06af5490-7d41-4e2b-8993-d8559a4e66c5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:59:02.915200 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06af5490-7d41-4e2b-8993-d8559a4e66c5 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:59:02.938707 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 943/1880] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Sep 1 14:59:02 2022] POST /identity/v3/users => generated 299 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:59:02.942754 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccefac04-e0f4-443e-945f-a32af4d86505 admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:59:02.954037 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccefac04-e0f4-443e-945f-a32af4d86505 admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:59:02.965948 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec05cc5-58f4-4515-949a-0372074a0d1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:59:02.966788 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec05cc5-58f4-4515-949a-0372074a0d1d admin admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:02.967170 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec05cc5-58f4-4515-949a-0372074a0d1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:02.967538 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec05cc5-58f4-4515-949a-0372074a0d1d admin admin] PATH_INFO: `/v3/roles` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:02.968285 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec05cc5-58f4-4515-949a-0372074a0d1d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:59:02.969976 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec05cc5-58f4-4515-949a-0372074a0d1d admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:59:02.984802 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 938/1881] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 1 14:59:02 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:59:02.986012 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06af5490-7d41-4e2b-8993-d8559a4e66c5 admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:59:02.996854 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06af5490-7d41-4e2b-8993-d8559a4e66c5 admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:59:03.009958 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa808e5-4c79-4167-9ad8-a8358fed9cc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:59:03.010765 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa808e5-4c79-4167-9ad8-a8358fed9cc7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:03.011049 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa808e5-4c79-4167-9ad8-a8358fed9cc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:03.011321 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa808e5-4c79-4167-9ad8-a8358fed9cc7 admin admin] PATH_INFO: `/v3/projects/da6a611cf16147dfa2f3d8c71a20ce85/users/8634d6b34fb14ddebf7263e68346700e/roles/2a2adb769e55476b82e5014897e0a9c0` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:03.027337 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa808e5-4c79-4167-9ad8-a8358fed9cc7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=da6a611cf16147dfa2f3d8c71a20ce85, user_id=8634d6b34fb14ddebf7263e68346700e, role_id=2a2adb769e55476b82e5014897e0a9c0)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:59:03.030107 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa808e5-4c79-4167-9ad8-a8358fed9cc7 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:59:03.044063 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 944/1882] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 1 14:59:02 2022] PUT /identity/v3/projects/da6a611cf16147dfa2f3d8c71a20ce85/users/8634d6b34fb14ddebf7263e68346700e/roles/2a2adb769e55476b82e5014897e0a9c0 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 14:59:03.048122 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec05cc5-58f4-4515-949a-0372074a0d1d admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:59:03.059211 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec05cc5-58f4-4515-949a-0372074a0d1d admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:59:03.120809 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9197f4-86c7-453f-9156-d6d781884ba6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:59:03.121433 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9197f4-86c7-453f-9156-d6d781884ba6 admin admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:03.122599 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9197f4-86c7-453f-9156-d6d781884ba6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:03.122935 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9197f4-86c7-453f-9156-d6d781884ba6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:03.123750 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9197f4-86c7-453f-9156-d6d781884ba6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:59:03.126204 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9197f4-86c7-453f-9156-d6d781884ba6 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:59:03.145651 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 939/1883] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 1 14:59:03 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:59:03.146658 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa808e5-4c79-4167-9ad8-a8358fed9cc7 admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:59:03.159387 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa808e5-4c79-4167-9ad8-a8358fed9cc7 admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:59:03.179600 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15afd715-2521-4d72-af73-5a570be8d3ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:59:03.180462 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15afd715-2521-4d72-af73-5a570be8d3ae admin admin] REQUEST_METHOD: `PUT` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:03.180557 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15afd715-2521-4d72-af73-5a570be8d3ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:03.180806 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15afd715-2521-4d72-af73-5a570be8d3ae admin admin] PATH_INFO: `/v3/projects/da6a611cf16147dfa2f3d8c71a20ce85/users/8634d6b34fb14ddebf7263e68346700e/roles/2a2adb769e55476b82e5014897e0a9c0` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:03.182796 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15afd715-2521-4d72-af73-5a570be8d3ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=da6a611cf16147dfa2f3d8c71a20ce85, user_id=8634d6b34fb14ddebf7263e68346700e, role_id=2a2adb769e55476b82e5014897e0a9c0)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:59:03.185428 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15afd715-2521-4d72-af73-5a570be8d3ae admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:59:03.202426 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 945/1884] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 1 14:59:03 2022] PUT /identity/v3/projects/da6a611cf16147dfa2f3d8c71a20ce85/users/8634d6b34fb14ddebf7263e68346700e/roles/2a2adb769e55476b82e5014897e0a9c0 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 14:59:03.213346 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a1bbd6-012f-42f2-8e36-3ab91d35f489 None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:03.213900 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a1bbd6-012f-42f2-8e36-3ab91d35f489 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:03.213900 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a1bbd6-012f-42f2-8e36-3ab91d35f489 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:03.242813 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-f5a1bbd6-012f-42f2-8e36-3ab91d35f489 None None] MFA Rules not processed for user `8634d6b34fb14ddebf7263e68346700e`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:59:03.244023 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-f5a1bbd6-012f-42f2-8e36-3ab91d35f489 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:59:03.276382 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:59:03.315421 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 940/1885] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 14:59:03 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:59:03.326647 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15afd715-2521-4d72-af73-5a570be8d3ae admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:59:03.328383 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-15afd715-2521-4d72-af73-5a570be8d3ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:59:03.338908 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:59:03.392083 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15afd715-2521-4d72-af73-5a570be8d3ae admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:59:03.396136 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3d8926-27d8-46ae-b5f1-41ee5cc2d386 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3b81739e34e4e648f88b6d73fcb5e46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:59:03.396735 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3d8926-27d8-46ae-b5f1-41ee5cc2d386 service cinder] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:03.397008 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3d8926-27d8-46ae-b5f1-41ee5cc2d386 service cinder] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:03.397270 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3d8926-27d8-46ae-b5f1-41ee5cc2d386 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:03.454608 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3d8926-27d8-46ae-b5f1-41ee5cc2d386 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:59:03.456076 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3d8926-27d8-46ae-b5f1-41ee5cc2d386 service cinder] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:59:03.465882 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 946/1886] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 14:59:03 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:59:07.262707 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a1bbd6-012f-42f2-8e36-3ab91d35f489 None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:59:07.264145 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-f5a1bbd6-012f-42f2-8e36-3ab91d35f489 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:59:07.277365 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a1bbd6-012f-42f2-8e36-3ab91d35f489 None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:59:07.281699 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6cc762-96cf-421b-a4b7-60d4068d2b28 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3b81739e34e4e648f88b6d73fcb5e46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:59:07.282345 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6cc762-96cf-421b-a4b7-60d4068d2b28 service cinder] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:07.282901 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6cc762-96cf-421b-a4b7-60d4068d2b28 service cinder] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:07.283191 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6cc762-96cf-421b-a4b7-60d4068d2b28 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:07.478361 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6cc762-96cf-421b-a4b7-60d4068d2b28 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:59:07.480652 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6cc762-96cf-421b-a4b7-60d4068d2b28 service cinder] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:59:07.512185 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 941/1887] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 14:59:07 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 250 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:59:08.095005 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3d8926-27d8-46ae-b5f1-41ee5cc2d386 service cinder] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:59:08.098510 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-6e3d8926-27d8-46ae-b5f1-41ee5cc2d386 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:59:08.212189 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3d8926-27d8-46ae-b5f1-41ee5cc2d386 service cinder] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:59:08.219464 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e7cc81-2128-40aa-b432-4140b3f88f19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:59:08.221724 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e7cc81-2128-40aa-b432-4140b3f88f19 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:08.222248 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e7cc81-2128-40aa-b432-4140b3f88f19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:08.223331 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e7cc81-2128-40aa-b432-4140b3f88f19 admin admin] PATH_INFO: `/v3/users/16e9b302197f4d75b07c42db666c71a8` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:08.226941 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e7cc81-2128-40aa-b432-4140b3f88f19 admin admin] RBAC: Authorizing `identity:delete_user(user_id=16e9b302197f4d75b07c42db666c71a8)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:59:08.234563 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e7cc81-2128-40aa-b432-4140b3f88f19 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:59:08.370774 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-78e7cc81-2128-40aa-b432-4140b3f88f19 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '16e9b302197f4d75b07c42db666c71a8'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:08.392721 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-78e7cc81-2128-40aa-b432-4140b3f88f19 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '16e9b302197f4d75b07c42db666c71a8'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:08.393653 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-78e7cc81-2128-40aa-b432-4140b3f88f19 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '16e9b302197f4d75b07c42db666c71a8'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:08.412345 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:59:08.429444 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-78e7cc81-2128-40aa-b432-4140b3f88f19 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '16e9b302197f4d75b07c42db666c71a8'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:08.430420 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-78e7cc81-2128-40aa-b432-4140b3f88f19 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '16e9b302197f4d75b07c42db666c71a8'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:08.499193 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 947/1888] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Sep 1 14:59:08 2022] DELETE /identity/v3/users/16e9b302197f4d75b07c42db666c71a8 => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:59:08.499803 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ba2649-a721-43c4-a879-21258876cd39 None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:08.500676 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ba2649-a721-43c4-a879-21258876cd39 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:08.501143 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ba2649-a721-43c4-a879-21258876cd39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:08.548832 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-d1ba2649-a721-43c4-a879-21258876cd39 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:59:08.553900 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-d1ba2649-a721-43c4-a879-21258876cd39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:59:08.764481 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 942/1889] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 14:59:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:59:08.775151 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e7cc81-2128-40aa-b432-4140b3f88f19 admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:59:08.777007 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-78e7cc81-2128-40aa-b432-4140b3f88f19 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:59:08.780504 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ba2649-a721-43c4-a879-21258876cd39 None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:59:08.781991 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-d1ba2649-a721-43c4-a879-21258876cd39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:59:08.891739 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ba2649-a721-43c4-a879-21258876cd39 None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:59:08.907284 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e7cc81-2128-40aa-b432-4140b3f88f19 admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:59:08.908583 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410bf7e9-f531-427b-88e4-dfd974a732c0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3b81739e34e4e648f88b6d73fcb5e46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:59:08.909388 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410bf7e9-f531-427b-88e4-dfd974a732c0 service cinder] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:08.910322 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410bf7e9-f531-427b-88e4-dfd974a732c0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:08.910598 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410bf7e9-f531-427b-88e4-dfd974a732c0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:08.914282 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfff7ba-41d2-4c90-983a-50f5a3a95675 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2790149639aa418bbc4e4982e93eda44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:59:08.914563 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfff7ba-41d2-4c90-983a-50f5a3a95675 service neutron] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:08.914830 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfff7ba-41d2-4c90-983a-50f5a3a95675 service neutron] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:08.915101 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfff7ba-41d2-4c90-983a-50f5a3a95675 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:08.917427 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-410bf7e9-f531-427b-88e4-dfd974a732c0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:59:08.934937 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfff7ba-41d2-4c90-983a-50f5a3a95675 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:59:08.939993 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfff7ba-41d2-4c90-983a-50f5a3a95675 service neutron] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:59:08.963400 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:59:08.980297 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 948/1890] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 14:59:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:59:09.078343 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410bf7e9-f531-427b-88e4-dfd974a732c0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:59:09.079827 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410bf7e9-f531-427b-88e4-dfd974a732c0 service cinder] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:59:09.107451 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 943/1891] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 14:59:08 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 14:59:09.531992 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfff7ba-41d2-4c90-983a-50f5a3a95675 service neutron] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:59:09.534147 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-cbfff7ba-41d2-4c90-983a-50f5a3a95675 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:59:09.557004 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfff7ba-41d2-4c90-983a-50f5a3a95675 service neutron] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:59:09.570885 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9855d4d-a415-4691-b87c-c2467b611386 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:59:09.572217 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9855d4d-a415-4691-b87c-c2467b611386 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:09.572792 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9855d4d-a415-4691-b87c-c2467b611386 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:09.573605 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9855d4d-a415-4691-b87c-c2467b611386 admin admin] PATH_INFO: `/v3/projects/f298a11bbc664dbe97b07c7f1a84d975` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:09.575278 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9855d4d-a415-4691-b87c-c2467b611386 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f298a11bbc664dbe97b07c7f1a84d975)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:59:09.580417 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9855d4d-a415-4691-b87c-c2467b611386 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:59:09.731105 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70be48f8-47db-4272-ac87-deafd419b4e8 None None] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:09.731658 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70be48f8-47db-4272-ac87-deafd419b4e8 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:09.731658 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70be48f8-47db-4272-ac87-deafd419b4e8 None None] PATH_INFO: `/` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:09.733273 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 944/1892] 10.222.0.33 () {54 vars in 921 bytes} [Thu Sep 1 14:59:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:59:09.740290 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-c9855d4d-a415-4691-b87c-c2467b611386 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f298a11bbc664dbe97b07c7f1a84d975', 'request_id': 'req-c9855d4d-a415-4691-b87c-c2467b611386'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:09.746042 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca12aa03-a8c5-48eb-a58c-5ee1cdd32f62 None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:09.746386 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca12aa03-a8c5-48eb-a58c-5ee1cdd32f62 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:09.746667 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca12aa03-a8c5-48eb-a58c-5ee1cdd32f62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:09.790021 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-c9855d4d-a415-4691-b87c-c2467b611386 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f298a11bbc664dbe97b07c7f1a84d975', 'request_id': 'req-c9855d4d-a415-4691-b87c-c2467b611386'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:09.811628 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-ca12aa03-a8c5-48eb-a58c-5ee1cdd32f62 None None] MFA Rules not processed for user `4858bb8621854ec8aba836e7d9050498`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:59:09.814929 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-ca12aa03-a8c5-48eb-a58c-5ee1cdd32f62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:59:09.819112 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-c9855d4d-a415-4691-b87c-c2467b611386 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f298a11bbc664dbe97b07c7f1a84d975', 'request_id': 'req-c9855d4d-a415-4691-b87c-c2467b611386'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:09.858585 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-c9855d4d-a415-4691-b87c-c2467b611386 admin admin] The token cache is being invalidate because project f298a11bbc664dbe97b07c7f1a84d975 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81342) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:59:09.858932 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-c9855d4d-a415-4691-b87c-c2467b611386 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:09.865418 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-c9855d4d-a415-4691-b87c-c2467b611386 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:09.874210 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 949/1893] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Sep 1 14:59:09 2022] DELETE /identity/v3/projects/f298a11bbc664dbe97b07c7f1a84d975 => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:59:09.875381 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9855d4d-a415-4691-b87c-c2467b611386 admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:59:09.877220 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-c9855d4d-a415-4691-b87c-c2467b611386 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:59:09.958776 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 945/1894] 10.222.0.33 () {58 vars in 1018 bytes} [Thu Sep 1 14:59:09 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 14:59:09.985733 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9855d4d-a415-4691-b87c-c2467b611386 admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:59:09.992099 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e0fe46-cb6f-460f-ace2-e48f559a9f5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:59:09.993243 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e0fe46-cb6f-460f-ace2-e48f559a9f5a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:09.993654 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e0fe46-cb6f-460f-ace2-e48f559a9f5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:09.994340 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e0fe46-cb6f-460f-ace2-e48f559a9f5a admin admin] PATH_INFO: `/v3/users/6f49ce0282bd41a3b85706c584453362` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:09.995689 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e0fe46-cb6f-460f-ace2-e48f559a9f5a admin admin] RBAC: Authorizing `identity:delete_user(user_id=6f49ce0282bd41a3b85706c584453362)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:59:09.998048 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e0fe46-cb6f-460f-ace2-e48f559a9f5a admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:59:10.015643 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:59:10.088483 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca12aa03-a8c5-48eb-a58c-5ee1cdd32f62 None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:59:10.090476 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-ca12aa03-a8c5-48eb-a58c-5ee1cdd32f62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:59:10.113717 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-c8e0fe46-cb6f-460f-ace2-e48f559a9f5a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6f49ce0282bd41a3b85706c584453362'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:10.116627 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca12aa03-a8c5-48eb-a58c-5ee1cdd32f62 None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:59:10.123485 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec2798d-240c-408a-828c-df1ca200eb29 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4858bb8621854ec8aba836e7d9050498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:59:10.124374 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-c8e0fe46-cb6f-460f-ace2-e48f559a9f5a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6f49ce0282bd41a3b85706c584453362'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:10.124509 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec2798d-240c-408a-828c-df1ca200eb29 service nova] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:10.124782 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec2798d-240c-408a-828c-df1ca200eb29 service nova] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:10.125220 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec2798d-240c-408a-828c-df1ca200eb29 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:10.125343 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-c8e0fe46-cb6f-460f-ace2-e48f559a9f5a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6f49ce0282bd41a3b85706c584453362'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:10.136181 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec2798d-240c-408a-828c-df1ca200eb29 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:59:10.141987 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec2798d-240c-408a-828c-df1ca200eb29 service nova] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:59:10.144671 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-c8e0fe46-cb6f-460f-ace2-e48f559a9f5a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6f49ce0282bd41a3b85706c584453362'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:10.145656 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-c8e0fe46-cb6f-460f-ace2-e48f559a9f5a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6f49ce0282bd41a3b85706c584453362'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:10.154490 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 946/1895] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 14:59:10 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 14:59:10.183781 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 950/1896] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Sep 1 14:59:09 2022] DELETE /identity/v3/users/6f49ce0282bd41a3b85706c584453362 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:59:10.842009 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec2798d-240c-408a-828c-df1ca200eb29 service nova] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:59:10.853710 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec2798d-240c-408a-828c-df1ca200eb29 service nova] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:59:10.918157 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf25a882-9b06-4886-8e95-d0d2b4ee9f1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:59:10.918815 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf25a882-9b06-4886-8e95-d0d2b4ee9f1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:10.919097 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf25a882-9b06-4886-8e95-d0d2b4ee9f1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:10.919365 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf25a882-9b06-4886-8e95-d0d2b4ee9f1b admin admin] PATH_INFO: `/v3/projects/aae5184a9e4c46e7b71a3c9c5516422b` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:10.920464 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf25a882-9b06-4886-8e95-d0d2b4ee9f1b admin admin] RBAC: Authorizing `identity:delete_project(project_id=aae5184a9e4c46e7b71a3c9c5516422b)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:59:10.922533 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf25a882-9b06-4886-8e95-d0d2b4ee9f1b admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:59:10.963037 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:59:10.971405 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-bf25a882-9b06-4886-8e95-d0d2b4ee9f1b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aae5184a9e4c46e7b71a3c9c5516422b', 'request_id': 'req-bf25a882-9b06-4886-8e95-d0d2b4ee9f1b'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:10.977455 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-bf25a882-9b06-4886-8e95-d0d2b4ee9f1b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aae5184a9e4c46e7b71a3c9c5516422b', 'request_id': 'req-bf25a882-9b06-4886-8e95-d0d2b4ee9f1b'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:10.985562 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-bf25a882-9b06-4886-8e95-d0d2b4ee9f1b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aae5184a9e4c46e7b71a3c9c5516422b', 'request_id': 'req-bf25a882-9b06-4886-8e95-d0d2b4ee9f1b'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:10.995704 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-bf25a882-9b06-4886-8e95-d0d2b4ee9f1b admin admin] The token cache is being invalidate because project aae5184a9e4c46e7b71a3c9c5516422b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81341) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:59:10.995989 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-bf25a882-9b06-4886-8e95-d0d2b4ee9f1b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:10.996620 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-bf25a882-9b06-4886-8e95-d0d2b4ee9f1b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:10.998101 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 947/1897] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Sep 1 14:59:10 2022] DELETE /identity/v3/projects/aae5184a9e4c46e7b71a3c9c5516422b => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:59:11.002176 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e0fe46-cb6f-460f-ace2-e48f559a9f5a admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:59:11.004132 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-c8e0fe46-cb6f-460f-ace2-e48f559a9f5a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:59:11.016865 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e0fe46-cb6f-460f-ace2-e48f559a9f5a admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:59:11.073064 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da7bc16-4b81-45e7-9df1-b1dee2829443 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:59:11.073692 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da7bc16-4b81-45e7-9df1-b1dee2829443 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:11.073973 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da7bc16-4b81-45e7-9df1-b1dee2829443 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:11.074272 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da7bc16-4b81-45e7-9df1-b1dee2829443 admin admin] PATH_INFO: `/v3/users/3d6583a2d5694b398f23efbefdc1098d` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:11.075402 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da7bc16-4b81-45e7-9df1-b1dee2829443 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3d6583a2d5694b398f23efbefdc1098d)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:59:11.077321 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da7bc16-4b81-45e7-9df1-b1dee2829443 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:59:11.127185 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-9da7bc16-4b81-45e7-9df1-b1dee2829443 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3d6583a2d5694b398f23efbefdc1098d'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:11.135554 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-9da7bc16-4b81-45e7-9df1-b1dee2829443 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3d6583a2d5694b398f23efbefdc1098d'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:11.136463 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-9da7bc16-4b81-45e7-9df1-b1dee2829443 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3d6583a2d5694b398f23efbefdc1098d'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:11.150253 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-9da7bc16-4b81-45e7-9df1-b1dee2829443 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3d6583a2d5694b398f23efbefdc1098d'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:11.150983 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-9da7bc16-4b81-45e7-9df1-b1dee2829443 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3d6583a2d5694b398f23efbefdc1098d'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:11.168569 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 951/1898] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Sep 1 14:59:11 2022] DELETE /identity/v3/users/3d6583a2d5694b398f23efbefdc1098d => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:59:11.911747 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf25a882-9b06-4886-8e95-d0d2b4ee9f1b admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:59:11.913567 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-bf25a882-9b06-4886-8e95-d0d2b4ee9f1b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:59:11.933828 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf25a882-9b06-4886-8e95-d0d2b4ee9f1b admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:59:12.048824 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33976f7d-9257-4598-8e9c-e52b0069ef41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:59:12.049779 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33976f7d-9257-4598-8e9c-e52b0069ef41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:12.052691 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33976f7d-9257-4598-8e9c-e52b0069ef41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:12.053220 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33976f7d-9257-4598-8e9c-e52b0069ef41 admin admin] PATH_INFO: `/v3/projects/4896268c46ee41549356477e4932e02d` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:12.056639 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33976f7d-9257-4598-8e9c-e52b0069ef41 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4896268c46ee41549356477e4932e02d)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:59:12.062148 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33976f7d-9257-4598-8e9c-e52b0069ef41 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:59:12.177693 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-33976f7d-9257-4598-8e9c-e52b0069ef41 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4896268c46ee41549356477e4932e02d', 'request_id': 'req-33976f7d-9257-4598-8e9c-e52b0069ef41'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:12.187706 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-33976f7d-9257-4598-8e9c-e52b0069ef41 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4896268c46ee41549356477e4932e02d', 'request_id': 'req-33976f7d-9257-4598-8e9c-e52b0069ef41'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:12.209554 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-33976f7d-9257-4598-8e9c-e52b0069ef41 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4896268c46ee41549356477e4932e02d', 'request_id': 'req-33976f7d-9257-4598-8e9c-e52b0069ef41'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:12.226024 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-33976f7d-9257-4598-8e9c-e52b0069ef41 admin admin] The token cache is being invalidate because project 4896268c46ee41549356477e4932e02d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81341) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:59:12.227068 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-33976f7d-9257-4598-8e9c-e52b0069ef41 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:12.229336 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-33976f7d-9257-4598-8e9c-e52b0069ef41 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:12.231783 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 948/1899] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Sep 1 14:59:11 2022] DELETE /identity/v3/projects/4896268c46ee41549356477e4932e02d => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:59:12.240302 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71feaac-0406-4f1f-a331-f546f8e232e0 None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:12.240894 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71feaac-0406-4f1f-a331-f546f8e232e0 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:12.243194 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71feaac-0406-4f1f-a331-f546f8e232e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:12.263917 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:59:12.286357 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-b71feaac-0406-4f1f-a331-f546f8e232e0 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:59:12.296267 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-b71feaac-0406-4f1f-a331-f546f8e232e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:59:12.504118 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 952/1900] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 14:59:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:59:12.517139 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ee37c6-091d-49ac-af13-a0ede1dd236b None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:12.517809 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ee37c6-091d-49ac-af13-a0ede1dd236b None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:12.518229 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ee37c6-091d-49ac-af13-a0ede1dd236b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:12.550955 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-b9ee37c6-091d-49ac-af13-a0ede1dd236b None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:59:12.552933 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-b9ee37c6-091d-49ac-af13-a0ede1dd236b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:59:12.567973 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 949/1901] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 14:59:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 14:59:12.570291 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71feaac-0406-4f1f-a331-f546f8e232e0 None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:59:12.583505 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71feaac-0406-4f1f-a331-f546f8e232e0 None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:59:12.590799 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7838d5-768f-4a4b-84c8-312b9b325639 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:59:12.591435 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7838d5-768f-4a4b-84c8-312b9b325639 admin admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:12.591713 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7838d5-768f-4a4b-84c8-312b9b325639 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:12.591981 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7838d5-768f-4a4b-84c8-312b9b325639 admin admin] PATH_INFO: `/v3/domains` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:12.592633 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7838d5-768f-4a4b-84c8-312b9b325639 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:59:12.592958 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7838d5-768f-4a4b-84c8-312b9b325639 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:59:12.594689 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7838d5-768f-4a4b-84c8-312b9b325639 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:59:12.607521 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 953/1902] 10.222.0.33 () {58 vars in 1153 bytes} [Thu Sep 1 14:59:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:59:12.612507 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ee37c6-091d-49ac-af13-a0ede1dd236b None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:59:12.629206 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ee37c6-091d-49ac-af13-a0ede1dd236b None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:59:12.641738 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be49eed8-53dd-4b9c-bc4c-aa9d19ab52eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:59:12.645294 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be49eed8-53dd-4b9c-bc4c-aa9d19ab52eb admin admin] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:12.645592 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be49eed8-53dd-4b9c-bc4c-aa9d19ab52eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:12.645961 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be49eed8-53dd-4b9c-bc4c-aa9d19ab52eb admin admin] PATH_INFO: `/v3/projects` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:12.658176 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be49eed8-53dd-4b9c-bc4c-aa9d19ab52eb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:59:12.658176 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be49eed8-53dd-4b9c-bc4c-aa9d19ab52eb admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:59:12.708514 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 950/1903] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Sep 1 14:59:12 2022] POST /identity/v3/projects => generated 385 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:59:12.712119 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7838d5-768f-4a4b-84c8-312b9b325639 admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:59:12.722277 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7838d5-768f-4a4b-84c8-312b9b325639 admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:59:12.787342 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232f953b-9ca2-497b-98d6-48df31fefed7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:59:12.788249 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232f953b-9ca2-497b-98d6-48df31fefed7 admin admin] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:12.788529 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232f953b-9ca2-497b-98d6-48df31fefed7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:12.788830 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232f953b-9ca2-497b-98d6-48df31fefed7 admin admin] PATH_INFO: `/v3/users` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:12.789473 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232f953b-9ca2-497b-98d6-48df31fefed7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:59:12.796705 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232f953b-9ca2-497b-98d6-48df31fefed7 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:59:12.830412 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 954/1904] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Sep 1 14:59:12 2022] POST /identity/v3/users => generated 293 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:59:12.831874 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be49eed8-53dd-4b9c-bc4c-aa9d19ab52eb admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:59:12.843198 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be49eed8-53dd-4b9c-bc4c-aa9d19ab52eb admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:59:12.847864 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdbaf45-26e1-49d6-a462-442f45166439 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:59:12.848461 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdbaf45-26e1-49d6-a462-442f45166439 admin admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:12.848741 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdbaf45-26e1-49d6-a462-442f45166439 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:12.849014 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdbaf45-26e1-49d6-a462-442f45166439 admin admin] PATH_INFO: `/v3/roles` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:12.849673 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afdbaf45-26e1-49d6-a462-442f45166439 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:59:12.851313 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afdbaf45-26e1-49d6-a462-442f45166439 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:59:12.861132 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 951/1905] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 1 14:59:12 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:59:12.865826 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232f953b-9ca2-497b-98d6-48df31fefed7 admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:59:12.875879 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232f953b-9ca2-497b-98d6-48df31fefed7 admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:59:12.881475 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c615fee-8320-4e27-b9ca-0dbe0f272e7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:59:12.882139 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c615fee-8320-4e27-b9ca-0dbe0f272e7f admin admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:12.882442 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c615fee-8320-4e27-b9ca-0dbe0f272e7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:12.882716 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c615fee-8320-4e27-b9ca-0dbe0f272e7f admin admin] PATH_INFO: `/v3/roles` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:12.883356 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c615fee-8320-4e27-b9ca-0dbe0f272e7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:59:12.885069 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c615fee-8320-4e27-b9ca-0dbe0f272e7f admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:59:12.895899 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 955/1906] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 1 14:59:12 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:59:12.899948 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdbaf45-26e1-49d6-a462-442f45166439 admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:59:12.911452 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdbaf45-26e1-49d6-a462-442f45166439 admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:59:12.917268 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1833bd7d-b9ff-4ac4-8b0e-c1f42cbdaf6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:59:12.917925 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1833bd7d-b9ff-4ac4-8b0e-c1f42cbdaf6e admin admin] REQUEST_METHOD: `PUT` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:12.918208 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1833bd7d-b9ff-4ac4-8b0e-c1f42cbdaf6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:12.918504 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1833bd7d-b9ff-4ac4-8b0e-c1f42cbdaf6e admin admin] PATH_INFO: `/v3/projects/8f96f009ba564b7993f50e5ca10a2cac/users/54047e52131e433990fac55f95b0e728/roles/5d1feef0e13d4874928bd86258c4e3b2` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:12.935399 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1833bd7d-b9ff-4ac4-8b0e-c1f42cbdaf6e admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f96f009ba564b7993f50e5ca10a2cac, user_id=54047e52131e433990fac55f95b0e728, role_id=5d1feef0e13d4874928bd86258c4e3b2)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:59:12.938348 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1833bd7d-b9ff-4ac4-8b0e-c1f42cbdaf6e admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:59:12.941721 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:59:12.955925 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 952/1907] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 1 14:59:12 2022] PUT /identity/v3/projects/8f96f009ba564b7993f50e5ca10a2cac/users/54047e52131e433990fac55f95b0e728/roles/5d1feef0e13d4874928bd86258c4e3b2 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:59:12.967135 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433022da-c116-4e58-8761-b6b3bf70ca99 None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:12.967539 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433022da-c116-4e58-8761-b6b3bf70ca99 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:12.967933 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433022da-c116-4e58-8761-b6b3bf70ca99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:12.998012 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-433022da-c116-4e58-8761-b6b3bf70ca99 None None] MFA Rules not processed for user `54047e52131e433990fac55f95b0e728`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:59:12.999269 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-433022da-c116-4e58-8761-b6b3bf70ca99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:59:13.068441 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:59:13.072953 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 956/1908] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 14:59:12 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:59:13.077427 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1833bd7d-b9ff-4ac4-8b0e-c1f42cbdaf6e admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:59:13.157851 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1833bd7d-b9ff-4ac4-8b0e-c1f42cbdaf6e admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:59:13.163682 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c583870a-3eb5-4c68-a902-9f37dab055df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:59:13.164285 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c583870a-3eb5-4c68-a902-9f37dab055df admin admin] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:13.164570 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c583870a-3eb5-4c68-a902-9f37dab055df admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:13.164845 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c583870a-3eb5-4c68-a902-9f37dab055df admin admin] PATH_INFO: `/v3/projects` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:13.165485 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c583870a-3eb5-4c68-a902-9f37dab055df admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:59:13.167461 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c583870a-3eb5-4c68-a902-9f37dab055df admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:59:13.200398 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433022da-c116-4e58-8761-b6b3bf70ca99 None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:59:13.208397 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 953/1909] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Sep 1 14:59:13 2022] POST /identity/v3/projects => generated 383 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:59:13.214505 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433022da-c116-4e58-8761-b6b3bf70ca99 None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:59:13.275281 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bf7441-13d9-49e8-8235-9c1a19ea78fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:59:13.275955 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bf7441-13d9-49e8-8235-9c1a19ea78fc admin admin] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:13.276151 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bf7441-13d9-49e8-8235-9c1a19ea78fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:13.276415 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bf7441-13d9-49e8-8235-9c1a19ea78fc admin admin] PATH_INFO: `/v3/users` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:13.277034 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bf7441-13d9-49e8-8235-9c1a19ea78fc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:59:13.279012 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bf7441-13d9-49e8-8235-9c1a19ea78fc admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:59:13.307320 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 957/1910] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Sep 1 14:59:13 2022] POST /identity/v3/users => generated 298 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:59:13.311358 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c583870a-3eb5-4c68-a902-9f37dab055df admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:59:13.321758 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c583870a-3eb5-4c68-a902-9f37dab055df admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:59:13.326448 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78ecfe5-2346-4af9-817c-d7d7d1972774 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:59:13.327048 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78ecfe5-2346-4af9-817c-d7d7d1972774 admin admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:13.327327 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78ecfe5-2346-4af9-817c-d7d7d1972774 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:13.327599 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78ecfe5-2346-4af9-817c-d7d7d1972774 admin admin] PATH_INFO: `/v3/roles` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:13.328247 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78ecfe5-2346-4af9-817c-d7d7d1972774 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:59:13.329855 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78ecfe5-2346-4af9-817c-d7d7d1972774 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:59:13.340380 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 954/1911] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 1 14:59:13 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:59:13.344510 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bf7441-13d9-49e8-8235-9c1a19ea78fc admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:59:13.354801 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bf7441-13d9-49e8-8235-9c1a19ea78fc admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:59:13.359352 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22775a48-6831-4529-a41f-8005e2c1daae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:59:13.359989 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22775a48-6831-4529-a41f-8005e2c1daae admin admin] REQUEST_METHOD: `PUT` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:13.360267 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22775a48-6831-4529-a41f-8005e2c1daae admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:13.360538 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22775a48-6831-4529-a41f-8005e2c1daae admin admin] PATH_INFO: `/v3/projects/0910b87ac4b3494aaf2a308e8a5d7bd7/users/1435b67048d1400597001f34ce44a316/roles/2a2adb769e55476b82e5014897e0a9c0` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:13.375675 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22775a48-6831-4529-a41f-8005e2c1daae admin admin] RBAC: Authorizing `identity:create_grant(project_id=0910b87ac4b3494aaf2a308e8a5d7bd7, user_id=1435b67048d1400597001f34ce44a316, role_id=2a2adb769e55476b82e5014897e0a9c0)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:59:13.378429 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22775a48-6831-4529-a41f-8005e2c1daae admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:59:13.392177 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 958/1912] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 1 14:59:13 2022] PUT /identity/v3/projects/0910b87ac4b3494aaf2a308e8a5d7bd7/users/1435b67048d1400597001f34ce44a316/roles/2a2adb769e55476b82e5014897e0a9c0 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 14:59:13.396221 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78ecfe5-2346-4af9-817c-d7d7d1972774 admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:59:13.406129 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78ecfe5-2346-4af9-817c-d7d7d1972774 admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:59:13.461982 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09daeb50-714d-40fe-b862-d788bd60910f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:59:13.463064 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09daeb50-714d-40fe-b862-d788bd60910f admin admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:13.463064 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09daeb50-714d-40fe-b862-d788bd60910f admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:13.463221 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09daeb50-714d-40fe-b862-d788bd60910f admin admin] PATH_INFO: `/v3/roles` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:13.465126 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09daeb50-714d-40fe-b862-d788bd60910f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:59:13.465987 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09daeb50-714d-40fe-b862-d788bd60910f admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:59:13.480621 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 955/1913] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 1 14:59:13 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:59:13.485195 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22775a48-6831-4529-a41f-8005e2c1daae admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:59:13.498085 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22775a48-6831-4529-a41f-8005e2c1daae admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:59:13.504636 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6891fd15-5eb8-47bb-90f8-8130d42eca03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:59:13.505720 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6891fd15-5eb8-47bb-90f8-8130d42eca03 admin admin] REQUEST_METHOD: `PUT` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:13.506074 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6891fd15-5eb8-47bb-90f8-8130d42eca03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:13.506506 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6891fd15-5eb8-47bb-90f8-8130d42eca03 admin admin] PATH_INFO: `/v3/projects/0910b87ac4b3494aaf2a308e8a5d7bd7/users/1435b67048d1400597001f34ce44a316/roles/2a2adb769e55476b82e5014897e0a9c0` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:13.508921 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6891fd15-5eb8-47bb-90f8-8130d42eca03 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0910b87ac4b3494aaf2a308e8a5d7bd7, user_id=1435b67048d1400597001f34ce44a316, role_id=2a2adb769e55476b82e5014897e0a9c0)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:59:13.511898 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6891fd15-5eb8-47bb-90f8-8130d42eca03 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:59:13.535089 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 959/1914] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 1 14:59:13 2022] PUT /identity/v3/projects/0910b87ac4b3494aaf2a308e8a5d7bd7/users/1435b67048d1400597001f34ce44a316/roles/2a2adb769e55476b82e5014897e0a9c0 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:59:13.540720 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d0b248-c4a4-47ac-ac22-e6737a4bbeb1 None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:13.541071 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d0b248-c4a4-47ac-ac22-e6737a4bbeb1 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:13.541405 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d0b248-c4a4-47ac-ac22-e6737a4bbeb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:13.580310 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-65d0b248-c4a4-47ac-ac22-e6737a4bbeb1 None None] MFA Rules not processed for user `1435b67048d1400597001f34ce44a316`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:59:13.582386 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-65d0b248-c4a4-47ac-ac22-e6737a4bbeb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:59:13.684644 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:59:13.715209 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 956/1915] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 14:59:13 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:59:13.760949 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6891fd15-5eb8-47bb-90f8-8130d42eca03 admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:59:13.763297 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-6891fd15-5eb8-47bb-90f8-8130d42eca03 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:59:13.868720 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6891fd15-5eb8-47bb-90f8-8130d42eca03 admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:59:13.873951 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c9f271-63db-4505-944a-28c6cf91a1eb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3b81739e34e4e648f88b6d73fcb5e46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:59:13.874701 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c9f271-63db-4505-944a-28c6cf91a1eb service cinder] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:13.875088 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c9f271-63db-4505-944a-28c6cf91a1eb service cinder] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:13.875419 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c9f271-63db-4505-944a-28c6cf91a1eb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:14.024108 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c9f271-63db-4505-944a-28c6cf91a1eb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:59:14.026683 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c9f271-63db-4505-944a-28c6cf91a1eb service cinder] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:59:14.068724 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 960/1916] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 14:59:13 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:59:14.167661 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d0b248-c4a4-47ac-ac22-e6737a4bbeb1 None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:59:14.169177 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-65d0b248-c4a4-47ac-ac22-e6737a4bbeb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:59:14.286823 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d0b248-c4a4-47ac-ac22-e6737a4bbeb1 None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:59:14.293531 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9671d4ee-6564-4e53-b40d-527a9ac9b267 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:59:14.294231 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9671d4ee-6564-4e53-b40d-527a9ac9b267 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:14.294548 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9671d4ee-6564-4e53-b40d-527a9ac9b267 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:14.294843 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9671d4ee-6564-4e53-b40d-527a9ac9b267 admin admin] PATH_INFO: `/v3/users/208e86a50cac462cb420ff0f08b9cbbc` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:14.297072 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9671d4ee-6564-4e53-b40d-527a9ac9b267 admin admin] RBAC: Authorizing `identity:delete_user(user_id=208e86a50cac462cb420ff0f08b9cbbc)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:59:14.299423 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9671d4ee-6564-4e53-b40d-527a9ac9b267 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:59:14.422501 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-9671d4ee-6564-4e53-b40d-527a9ac9b267 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '208e86a50cac462cb420ff0f08b9cbbc'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:14.431319 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-9671d4ee-6564-4e53-b40d-527a9ac9b267 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '208e86a50cac462cb420ff0f08b9cbbc'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:14.432097 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-9671d4ee-6564-4e53-b40d-527a9ac9b267 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '208e86a50cac462cb420ff0f08b9cbbc'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:14.445958 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-9671d4ee-6564-4e53-b40d-527a9ac9b267 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '208e86a50cac462cb420ff0f08b9cbbc'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:14.446850 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-9671d4ee-6564-4e53-b40d-527a9ac9b267 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '208e86a50cac462cb420ff0f08b9cbbc'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:14.461910 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 957/1917] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Sep 1 14:59:14 2022] DELETE /identity/v3/users/208e86a50cac462cb420ff0f08b9cbbc => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:59:14.466693 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c79df8-4450-4a6a-bdb0-93a293a9c0c1 None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:14.467215 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c79df8-4450-4a6a-bdb0-93a293a9c0c1 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:14.467465 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c79df8-4450-4a6a-bdb0-93a293a9c0c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:14.481309 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:59:14.488297 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-90c79df8-4450-4a6a-bdb0-93a293a9c0c1 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:59:14.489790 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-90c79df8-4450-4a6a-bdb0-93a293a9c0c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:59:14.594373 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 961/1918] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 14:59:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 14:59:14.594711 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9671d4ee-6564-4e53-b40d-527a9ac9b267 admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:59:14.596150 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-9671d4ee-6564-4e53-b40d-527a9ac9b267 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:59:14.668971 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9671d4ee-6564-4e53-b40d-527a9ac9b267 admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:59:14.672653 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3e8057-7fe2-4377-a84e-ab7aa7c52a9f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2790149639aa418bbc4e4982e93eda44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:59:14.673527 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3e8057-7fe2-4377-a84e-ab7aa7c52a9f service neutron] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:14.673812 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3e8057-7fe2-4377-a84e-ab7aa7c52a9f service neutron] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:14.674151 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3e8057-7fe2-4377-a84e-ab7aa7c52a9f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:14.684873 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3e8057-7fe2-4377-a84e-ab7aa7c52a9f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:59:14.686373 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3e8057-7fe2-4377-a84e-ab7aa7c52a9f service neutron] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:59:14.695620 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 958/1919] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 14:59:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 14:59:14.955336 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c79df8-4450-4a6a-bdb0-93a293a9c0c1 None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:59:14.957694 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-90c79df8-4450-4a6a-bdb0-93a293a9c0c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:59:14.970492 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c79df8-4450-4a6a-bdb0-93a293a9c0c1 None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:59:14.978103 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8021669a-c0f7-4d69-8a3b-56706aabafe8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:59:14.981252 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8021669a-c0f7-4d69-8a3b-56706aabafe8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:14.981542 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8021669a-c0f7-4d69-8a3b-56706aabafe8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:14.981815 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8021669a-c0f7-4d69-8a3b-56706aabafe8 admin admin] PATH_INFO: `/v3/projects/56203d292bbb4cb2bae5bc2f3a34d080` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:14.983873 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8021669a-c0f7-4d69-8a3b-56706aabafe8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=56203d292bbb4cb2bae5bc2f3a34d080)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:59:14.985563 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8021669a-c0f7-4d69-8a3b-56706aabafe8 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:59:15.035863 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-8021669a-c0f7-4d69-8a3b-56706aabafe8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '56203d292bbb4cb2bae5bc2f3a34d080', 'request_id': 'req-8021669a-c0f7-4d69-8a3b-56706aabafe8'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:15.042935 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-8021669a-c0f7-4d69-8a3b-56706aabafe8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '56203d292bbb4cb2bae5bc2f3a34d080', 'request_id': 'req-8021669a-c0f7-4d69-8a3b-56706aabafe8'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:15.051414 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-8021669a-c0f7-4d69-8a3b-56706aabafe8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '56203d292bbb4cb2bae5bc2f3a34d080', 'request_id': 'req-8021669a-c0f7-4d69-8a3b-56706aabafe8'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:15.062124 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-8021669a-c0f7-4d69-8a3b-56706aabafe8 admin admin] The token cache is being invalidate because project 56203d292bbb4cb2bae5bc2f3a34d080 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81342) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:59:15.062445 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-8021669a-c0f7-4d69-8a3b-56706aabafe8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:15.063151 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-8021669a-c0f7-4d69-8a3b-56706aabafe8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:15.064748 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 962/1920] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Sep 1 14:59:14 2022] DELETE /identity/v3/projects/56203d292bbb4cb2bae5bc2f3a34d080 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:59:15.068636 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3e8057-7fe2-4377-a84e-ab7aa7c52a9f service neutron] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:59:15.078564 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3e8057-7fe2-4377-a84e-ab7aa7c52a9f service neutron] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:59:15.086660 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:59:15.142442 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98388b47-332d-479f-850d-25a51b191080 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:59:15.142976 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98388b47-332d-479f-850d-25a51b191080 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:15.143223 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98388b47-332d-479f-850d-25a51b191080 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:15.143501 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98388b47-332d-479f-850d-25a51b191080 admin admin] PATH_INFO: `/v3/users/93ddb476fc534556a9bc37ddcc9a5c12` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:15.145141 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98388b47-332d-479f-850d-25a51b191080 admin admin] RBAC: Authorizing `identity:delete_user(user_id=93ddb476fc534556a9bc37ddcc9a5c12)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:59:15.147093 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98388b47-332d-479f-850d-25a51b191080 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:59:15.194987 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-98388b47-332d-479f-850d-25a51b191080 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '93ddb476fc534556a9bc37ddcc9a5c12'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:15.203617 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-98388b47-332d-479f-850d-25a51b191080 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '93ddb476fc534556a9bc37ddcc9a5c12'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:15.204231 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-98388b47-332d-479f-850d-25a51b191080 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '93ddb476fc534556a9bc37ddcc9a5c12'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:15.217508 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-98388b47-332d-479f-850d-25a51b191080 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '93ddb476fc534556a9bc37ddcc9a5c12'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:15.218312 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-98388b47-332d-479f-850d-25a51b191080 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '93ddb476fc534556a9bc37ddcc9a5c12'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:15.231417 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 959/1921] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Sep 1 14:59:15 2022] DELETE /identity/v3/users/93ddb476fc534556a9bc37ddcc9a5c12 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:59:15.500277 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8021669a-c0f7-4d69-8a3b-56706aabafe8 admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:59:15.501803 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-8021669a-c0f7-4d69-8a3b-56706aabafe8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:59:15.517712 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8021669a-c0f7-4d69-8a3b-56706aabafe8 admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:59:15.584904 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4855cdb-7a19-494c-9796-81109cc9a370 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:59:15.585540 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4855cdb-7a19-494c-9796-81109cc9a370 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:15.585819 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4855cdb-7a19-494c-9796-81109cc9a370 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:15.586088 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4855cdb-7a19-494c-9796-81109cc9a370 admin admin] PATH_INFO: `/v3/projects/69f76f3d089e433e85311d7ceb2ee087` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:15.587257 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4855cdb-7a19-494c-9796-81109cc9a370 admin admin] RBAC: Authorizing `identity:delete_project(project_id=69f76f3d089e433e85311d7ceb2ee087)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:59:15.589192 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4855cdb-7a19-494c-9796-81109cc9a370 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:59:15.631545 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:59:15.649916 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-f4855cdb-7a19-494c-9796-81109cc9a370 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '69f76f3d089e433e85311d7ceb2ee087', 'request_id': 'req-f4855cdb-7a19-494c-9796-81109cc9a370'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:15.657355 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-f4855cdb-7a19-494c-9796-81109cc9a370 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '69f76f3d089e433e85311d7ceb2ee087', 'request_id': 'req-f4855cdb-7a19-494c-9796-81109cc9a370'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:15.665476 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-f4855cdb-7a19-494c-9796-81109cc9a370 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '69f76f3d089e433e85311d7ceb2ee087', 'request_id': 'req-f4855cdb-7a19-494c-9796-81109cc9a370'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:15.681203 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-f4855cdb-7a19-494c-9796-81109cc9a370 admin admin] The token cache is being invalidate because project 69f76f3d089e433e85311d7ceb2ee087 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81342) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:59:15.682084 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-f4855cdb-7a19-494c-9796-81109cc9a370 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:15.683249 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-f4855cdb-7a19-494c-9796-81109cc9a370 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:15.685395 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 963/1922] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Sep 1 14:59:15 2022] DELETE /identity/v3/projects/69f76f3d089e433e85311d7ceb2ee087 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:59:15.689439 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98388b47-332d-479f-850d-25a51b191080 admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:59:15.691544 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-98388b47-332d-479f-850d-25a51b191080 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:59:15.705056 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98388b47-332d-479f-850d-25a51b191080 admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:59:15.764580 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922c07ca-2b28-4064-a505-e55b8b35c8e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:59:15.765290 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922c07ca-2b28-4064-a505-e55b8b35c8e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:15.765654 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922c07ca-2b28-4064-a505-e55b8b35c8e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:15.765993 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922c07ca-2b28-4064-a505-e55b8b35c8e7 admin admin] PATH_INFO: `/v3/users/8634d6b34fb14ddebf7263e68346700e` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:15.767211 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922c07ca-2b28-4064-a505-e55b8b35c8e7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8634d6b34fb14ddebf7263e68346700e)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:59:15.769823 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922c07ca-2b28-4064-a505-e55b8b35c8e7 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:59:15.804794 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:59:15.819568 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-922c07ca-2b28-4064-a505-e55b8b35c8e7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8634d6b34fb14ddebf7263e68346700e'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:15.826514 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-922c07ca-2b28-4064-a505-e55b8b35c8e7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8634d6b34fb14ddebf7263e68346700e'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:15.827134 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-922c07ca-2b28-4064-a505-e55b8b35c8e7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8634d6b34fb14ddebf7263e68346700e'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:15.839081 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-922c07ca-2b28-4064-a505-e55b8b35c8e7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8634d6b34fb14ddebf7263e68346700e'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:15.839710 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-922c07ca-2b28-4064-a505-e55b8b35c8e7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8634d6b34fb14ddebf7263e68346700e'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:15.851525 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 960/1923] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Sep 1 14:59:15 2022] DELETE /identity/v3/users/8634d6b34fb14ddebf7263e68346700e => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:59:16.114410 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4855cdb-7a19-494c-9796-81109cc9a370 admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:59:16.115968 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-f4855cdb-7a19-494c-9796-81109cc9a370 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:59:16.129591 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4855cdb-7a19-494c-9796-81109cc9a370 admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:59:16.193984 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71ad9a0-31c9-43d7-a611-4658f65c6da6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:59:16.194577 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71ad9a0-31c9-43d7-a611-4658f65c6da6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:16.194855 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71ad9a0-31c9-43d7-a611-4658f65c6da6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:16.195116 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71ad9a0-31c9-43d7-a611-4658f65c6da6 admin admin] PATH_INFO: `/v3/projects/da6a611cf16147dfa2f3d8c71a20ce85` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:16.196230 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71ad9a0-31c9-43d7-a611-4658f65c6da6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=da6a611cf16147dfa2f3d8c71a20ce85)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:59:16.198124 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71ad9a0-31c9-43d7-a611-4658f65c6da6 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:59:16.249979 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-e71ad9a0-31c9-43d7-a611-4658f65c6da6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'da6a611cf16147dfa2f3d8c71a20ce85', 'request_id': 'req-e71ad9a0-31c9-43d7-a611-4658f65c6da6'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:16.256219 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-e71ad9a0-31c9-43d7-a611-4658f65c6da6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'da6a611cf16147dfa2f3d8c71a20ce85', 'request_id': 'req-e71ad9a0-31c9-43d7-a611-4658f65c6da6'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:16.264252 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-e71ad9a0-31c9-43d7-a611-4658f65c6da6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'da6a611cf16147dfa2f3d8c71a20ce85', 'request_id': 'req-e71ad9a0-31c9-43d7-a611-4658f65c6da6'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:16.274718 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-e71ad9a0-31c9-43d7-a611-4658f65c6da6 admin admin] The token cache is being invalidate because project da6a611cf16147dfa2f3d8c71a20ce85 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81342) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:59:16.275005 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-e71ad9a0-31c9-43d7-a611-4658f65c6da6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:16.275617 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-e71ad9a0-31c9-43d7-a611-4658f65c6da6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:16.277078 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 964/1924] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Sep 1 14:59:16 2022] DELETE /identity/v3/projects/da6a611cf16147dfa2f3d8c71a20ce85 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:59:16.281851 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af2cbc2-3ea6-4ac0-b372-ebe8dfc2d7f7 None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:16.282356 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af2cbc2-3ea6-4ac0-b372-ebe8dfc2d7f7 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:16.282452 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af2cbc2-3ea6-4ac0-b372-ebe8dfc2d7f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:16.303629 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-2af2cbc2-3ea6-4ac0-b372-ebe8dfc2d7f7 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:59:16.304861 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-2af2cbc2-3ea6-4ac0-b372-ebe8dfc2d7f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:59:16.362500 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 961/1925] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 14:59:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:59:16.370559 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cb6db5-af2f-493d-85ea-511e9595b00d None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:16.370844 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cb6db5-af2f-493d-85ea-511e9595b00d None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:16.371114 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cb6db5-af2f-493d-85ea-511e9595b00d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:16.391009 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-84cb6db5-af2f-493d-85ea-511e9595b00d None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:59:16.392238 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-84cb6db5-af2f-493d-85ea-511e9595b00d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:59:16.400149 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 965/1926] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 14:59:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:59:16.404764 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af2cbc2-3ea6-4ac0-b372-ebe8dfc2d7f7 None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:59:16.415096 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af2cbc2-3ea6-4ac0-b372-ebe8dfc2d7f7 None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:59:16.419736 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a45bdb6-63b4-45c5-a6e4-866d81897099 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:59:16.420368 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a45bdb6-63b4-45c5-a6e4-866d81897099 admin admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:16.420646 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a45bdb6-63b4-45c5-a6e4-866d81897099 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:16.420916 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a45bdb6-63b4-45c5-a6e4-866d81897099 admin admin] PATH_INFO: `/v3/domains` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:16.421564 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a45bdb6-63b4-45c5-a6e4-866d81897099 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81341) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:59:16.421864 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a45bdb6-63b4-45c5-a6e4-866d81897099 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:59:16.423600 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a45bdb6-63b4-45c5-a6e4-866d81897099 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:59:16.436725 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 962/1927] 10.222.0.33 () {58 vars in 1153 bytes} [Thu Sep 1 14:59:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:59:16.441181 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cb6db5-af2f-493d-85ea-511e9595b00d None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:59:16.450584 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cb6db5-af2f-493d-85ea-511e9595b00d None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:59:16.455220 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a53530-c99f-4c21-9fa3-9722958a9706 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:59:16.455809 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a53530-c99f-4c21-9fa3-9722958a9706 admin admin] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:16.456088 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a53530-c99f-4c21-9fa3-9722958a9706 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:16.456411 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a53530-c99f-4c21-9fa3-9722958a9706 admin admin] PATH_INFO: `/v3/projects` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:16.457051 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a53530-c99f-4c21-9fa3-9722958a9706 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:59:16.459308 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a53530-c99f-4c21-9fa3-9722958a9706 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:59:16.486266 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 966/1928] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Sep 1 14:59:16 2022] POST /identity/v3/projects => generated 363 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:59:16.490997 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a45bdb6-63b4-45c5-a6e4-866d81897099 admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:59:16.500619 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a45bdb6-63b4-45c5-a6e4-866d81897099 admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:59:16.556928 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39cb35f-97e3-4041-9eda-ed5d8ac4e956 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:59:16.557533 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39cb35f-97e3-4041-9eda-ed5d8ac4e956 admin admin] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:16.557827 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39cb35f-97e3-4041-9eda-ed5d8ac4e956 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:16.558066 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39cb35f-97e3-4041-9eda-ed5d8ac4e956 admin admin] PATH_INFO: `/v3/users` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:16.558721 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39cb35f-97e3-4041-9eda-ed5d8ac4e956 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:59:16.560634 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39cb35f-97e3-4041-9eda-ed5d8ac4e956 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:59:16.585527 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 963/1929] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Sep 1 14:59:16 2022] POST /identity/v3/users => generated 282 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:59:16.586798 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a53530-c99f-4c21-9fa3-9722958a9706 admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:59:16.598122 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a53530-c99f-4c21-9fa3-9722958a9706 admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:59:16.602844 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b60bfb7-fa3e-4a67-a8c9-64491646a877 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:59:16.603440 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b60bfb7-fa3e-4a67-a8c9-64491646a877 admin admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:16.603718 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b60bfb7-fa3e-4a67-a8c9-64491646a877 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:16.603982 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b60bfb7-fa3e-4a67-a8c9-64491646a877 admin admin] PATH_INFO: `/v3/roles` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:16.604655 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b60bfb7-fa3e-4a67-a8c9-64491646a877 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:59:16.606258 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b60bfb7-fa3e-4a67-a8c9-64491646a877 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:59:16.616695 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 967/1930] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 1 14:59:16 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:59:16.620813 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39cb35f-97e3-4041-9eda-ed5d8ac4e956 admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:59:16.630875 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39cb35f-97e3-4041-9eda-ed5d8ac4e956 admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:59:16.635478 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c3b6c4-fbd6-48c5-aabf-d6f29fa34f73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:59:16.636115 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c3b6c4-fbd6-48c5-aabf-d6f29fa34f73 admin admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:16.636389 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c3b6c4-fbd6-48c5-aabf-d6f29fa34f73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:16.636731 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c3b6c4-fbd6-48c5-aabf-d6f29fa34f73 admin admin] PATH_INFO: `/v3/roles` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:16.637542 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c3b6c4-fbd6-48c5-aabf-d6f29fa34f73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:59:16.639289 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c3b6c4-fbd6-48c5-aabf-d6f29fa34f73 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:59:16.650555 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 964/1931] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 1 14:59:16 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:59:16.656265 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b60bfb7-fa3e-4a67-a8c9-64491646a877 admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:59:16.674421 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b60bfb7-fa3e-4a67-a8c9-64491646a877 admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:59:16.679821 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51474c72-43c7-4614-b881-8cb84a020e6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:59:16.680483 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51474c72-43c7-4614-b881-8cb84a020e6e admin admin] REQUEST_METHOD: `PUT` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:16.680762 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51474c72-43c7-4614-b881-8cb84a020e6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:16.681035 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51474c72-43c7-4614-b881-8cb84a020e6e admin admin] PATH_INFO: `/v3/projects/3cde10dae67b460abbc07bc3936f8143/users/ee1ea25915c44868a4e813a471abfc1c/roles/5d1feef0e13d4874928bd86258c4e3b2` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:16.690756 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:59:16.701031 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51474c72-43c7-4614-b881-8cb84a020e6e admin admin] RBAC: Authorizing `identity:create_grant(project_id=3cde10dae67b460abbc07bc3936f8143, user_id=ee1ea25915c44868a4e813a471abfc1c, role_id=5d1feef0e13d4874928bd86258c4e3b2)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:59:16.703714 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51474c72-43c7-4614-b881-8cb84a020e6e admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:59:16.736825 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 968/1932] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 1 14:59:16 2022] PUT /identity/v3/projects/3cde10dae67b460abbc07bc3936f8143/users/ee1ea25915c44868a4e813a471abfc1c/roles/5d1feef0e13d4874928bd86258c4e3b2 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:59:16.748466 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113470d0-4654-4114-a057-06d88b71d61d None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:16.748750 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113470d0-4654-4114-a057-06d88b71d61d None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:16.749082 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113470d0-4654-4114-a057-06d88b71d61d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:16.777083 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-113470d0-4654-4114-a057-06d88b71d61d None None] MFA Rules not processed for user `ee1ea25915c44868a4e813a471abfc1c`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:59:16.778325 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-113470d0-4654-4114-a057-06d88b71d61d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:59:16.782559 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:59:16.847717 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 965/1933] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 14:59:16 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:59:16.860368 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51474c72-43c7-4614-b881-8cb84a020e6e admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:59:16.862051 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-51474c72-43c7-4614-b881-8cb84a020e6e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:59:16.919512 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51474c72-43c7-4614-b881-8cb84a020e6e admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:59:16.923623 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a390e308-eece-4d93-a046-9339bf0016cf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3b81739e34e4e648f88b6d73fcb5e46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:59:16.923811 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a390e308-eece-4d93-a046-9339bf0016cf service cinder] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:16.924528 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a390e308-eece-4d93-a046-9339bf0016cf service cinder] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:16.924528 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a390e308-eece-4d93-a046-9339bf0016cf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:16.934436 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a390e308-eece-4d93-a046-9339bf0016cf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:59:16.935140 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a390e308-eece-4d93-a046-9339bf0016cf service cinder] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:59:16.944587 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 969/1934] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 14:59:16 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:59:20.542162 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113470d0-4654-4114-a057-06d88b71d61d None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:59:20.544269 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-113470d0-4654-4114-a057-06d88b71d61d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:59:20.628737 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113470d0-4654-4114-a057-06d88b71d61d None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:59:20.633139 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595a376e-3ffa-4a24-82e1-67aa86cdb6d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:59:20.633790 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595a376e-3ffa-4a24-82e1-67aa86cdb6d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:20.634070 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595a376e-3ffa-4a24-82e1-67aa86cdb6d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:20.634366 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595a376e-3ffa-4a24-82e1-67aa86cdb6d3 admin admin] PATH_INFO: `/v3/users/cdd15ae9e0944062bfbe8acb18433b9f` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:20.635474 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595a376e-3ffa-4a24-82e1-67aa86cdb6d3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cdd15ae9e0944062bfbe8acb18433b9f)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:59:20.637518 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595a376e-3ffa-4a24-82e1-67aa86cdb6d3 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:59:20.688808 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-595a376e-3ffa-4a24-82e1-67aa86cdb6d3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cdd15ae9e0944062bfbe8acb18433b9f'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:20.697407 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-595a376e-3ffa-4a24-82e1-67aa86cdb6d3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cdd15ae9e0944062bfbe8acb18433b9f'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:20.698168 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-595a376e-3ffa-4a24-82e1-67aa86cdb6d3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cdd15ae9e0944062bfbe8acb18433b9f'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:20.711982 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-595a376e-3ffa-4a24-82e1-67aa86cdb6d3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cdd15ae9e0944062bfbe8acb18433b9f'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:20.712673 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-595a376e-3ffa-4a24-82e1-67aa86cdb6d3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cdd15ae9e0944062bfbe8acb18433b9f'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:20.725287 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 966/1935] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Sep 1 14:59:20 2022] DELETE /identity/v3/users/cdd15ae9e0944062bfbe8acb18433b9f => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:59:20.729897 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee2e9e7-d3c3-433c-8e1d-35f6533e1f78 None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:20.730756 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee2e9e7-d3c3-433c-8e1d-35f6533e1f78 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:20.731148 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee2e9e7-d3c3-433c-8e1d-35f6533e1f78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:20.759689 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-0ee2e9e7-d3c3-433c-8e1d-35f6533e1f78 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:59:20.760598 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-0ee2e9e7-d3c3-433c-8e1d-35f6533e1f78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:59:20.832514 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 970/1936] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 14:59:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 14:59:20.844597 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595a376e-3ffa-4a24-82e1-67aa86cdb6d3 admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:59:20.846160 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-595a376e-3ffa-4a24-82e1-67aa86cdb6d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:59:20.879775 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:59:20.930584 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595a376e-3ffa-4a24-82e1-67aa86cdb6d3 admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:59:20.934819 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdd270d-d8cd-4fd6-9e2e-c6a4d29e6ef4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2790149639aa418bbc4e4982e93eda44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:59:20.935567 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdd270d-d8cd-4fd6-9e2e-c6a4d29e6ef4 service neutron] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:20.935975 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdd270d-d8cd-4fd6-9e2e-c6a4d29e6ef4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:20.936248 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdd270d-d8cd-4fd6-9e2e-c6a4d29e6ef4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:20.946926 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdd270d-d8cd-4fd6-9e2e-c6a4d29e6ef4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:59:20.948487 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdd270d-d8cd-4fd6-9e2e-c6a4d29e6ef4 service neutron] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:59:20.961210 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 967/1937] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 14:59:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:59:21.031771 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee2e9e7-d3c3-433c-8e1d-35f6533e1f78 None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:59:21.033393 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-0ee2e9e7-d3c3-433c-8e1d-35f6533e1f78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:59:21.168128 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee2e9e7-d3c3-433c-8e1d-35f6533e1f78 None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:59:21.173127 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c27fcd-a1f4-4fc8-914a-6ef967fac1f5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3b81739e34e4e648f88b6d73fcb5e46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:59:21.174125 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c27fcd-a1f4-4fc8-914a-6ef967fac1f5 service cinder] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:21.174770 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c27fcd-a1f4-4fc8-914a-6ef967fac1f5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:21.175722 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c27fcd-a1f4-4fc8-914a-6ef967fac1f5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:21.178656 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-82c27fcd-a1f4-4fc8-914a-6ef967fac1f5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:59:21.265230 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:59:21.291783 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c27fcd-a1f4-4fc8-914a-6ef967fac1f5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:59:21.294857 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c27fcd-a1f4-4fc8-914a-6ef967fac1f5 service cinder] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:59:21.313589 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 971/1938] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 14:59:21 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 282 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:59:21.429943 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdd270d-d8cd-4fd6-9e2e-c6a4d29e6ef4 service neutron] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:59:21.432317 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-ffdd270d-d8cd-4fd6-9e2e-c6a4d29e6ef4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:59:21.449197 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdd270d-d8cd-4fd6-9e2e-c6a4d29e6ef4 service neutron] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:59:21.454223 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b921a2-7d28-4305-816c-f489abd2a291 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:59:21.454943 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b921a2-7d28-4305-816c-f489abd2a291 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:21.455286 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b921a2-7d28-4305-816c-f489abd2a291 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:21.456210 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b921a2-7d28-4305-816c-f489abd2a291 admin admin] PATH_INFO: `/v3/projects/5fdf47652b8942bf8e80c8d229ca1896` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:21.457324 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b921a2-7d28-4305-816c-f489abd2a291 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5fdf47652b8942bf8e80c8d229ca1896)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:59:21.459664 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b921a2-7d28-4305-816c-f489abd2a291 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:59:21.537141 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-19b921a2-7d28-4305-816c-f489abd2a291 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5fdf47652b8942bf8e80c8d229ca1896', 'request_id': 'req-19b921a2-7d28-4305-816c-f489abd2a291'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:21.544856 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-19b921a2-7d28-4305-816c-f489abd2a291 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5fdf47652b8942bf8e80c8d229ca1896', 'request_id': 'req-19b921a2-7d28-4305-816c-f489abd2a291'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:21.554454 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-19b921a2-7d28-4305-816c-f489abd2a291 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5fdf47652b8942bf8e80c8d229ca1896', 'request_id': 'req-19b921a2-7d28-4305-816c-f489abd2a291'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:21.566789 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-19b921a2-7d28-4305-816c-f489abd2a291 admin admin] The token cache is being invalidate because project 5fdf47652b8942bf8e80c8d229ca1896 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81341) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:59:21.567076 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-19b921a2-7d28-4305-816c-f489abd2a291 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:21.567737 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-19b921a2-7d28-4305-816c-f489abd2a291 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:21.569521 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 968/1939] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Sep 1 14:59:21 2022] DELETE /identity/v3/projects/5fdf47652b8942bf8e80c8d229ca1896 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:59:21.573824 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c27fcd-a1f4-4fc8-914a-6ef967fac1f5 service cinder] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:59:21.585694 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c27fcd-a1f4-4fc8-914a-6ef967fac1f5 service cinder] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:59:21.664433 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae3f7cb-f5bb-4afe-9b31-09f1207db4b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:59:21.665097 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae3f7cb-f5bb-4afe-9b31-09f1207db4b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:21.665557 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae3f7cb-f5bb-4afe-9b31-09f1207db4b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:21.665929 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae3f7cb-f5bb-4afe-9b31-09f1207db4b8 admin admin] PATH_INFO: `/v3/users/d4384b5f37614a23b12309a063d6d40a` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:21.667592 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae3f7cb-f5bb-4afe-9b31-09f1207db4b8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d4384b5f37614a23b12309a063d6d40a)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:59:21.669933 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae3f7cb-f5bb-4afe-9b31-09f1207db4b8 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:59:21.728592 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-cae3f7cb-f5bb-4afe-9b31-09f1207db4b8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd4384b5f37614a23b12309a063d6d40a'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:21.739033 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-cae3f7cb-f5bb-4afe-9b31-09f1207db4b8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd4384b5f37614a23b12309a063d6d40a'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:21.739877 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-cae3f7cb-f5bb-4afe-9b31-09f1207db4b8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd4384b5f37614a23b12309a063d6d40a'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:21.756139 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-cae3f7cb-f5bb-4afe-9b31-09f1207db4b8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd4384b5f37614a23b12309a063d6d40a'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:21.757257 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-cae3f7cb-f5bb-4afe-9b31-09f1207db4b8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd4384b5f37614a23b12309a063d6d40a'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:21.771152 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 972/1940] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Sep 1 14:59:21 2022] DELETE /identity/v3/users/d4384b5f37614a23b12309a063d6d40a => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:59:22.340898 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b921a2-7d28-4305-816c-f489abd2a291 admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:59:22.342496 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-19b921a2-7d28-4305-816c-f489abd2a291 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:59:22.355812 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b921a2-7d28-4305-816c-f489abd2a291 admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:59:22.427761 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ee7bec-2c85-48bd-80a3-1a427cb2cc0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:59:22.428796 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ee7bec-2c85-48bd-80a3-1a427cb2cc0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:22.429075 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ee7bec-2c85-48bd-80a3-1a427cb2cc0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:22.429348 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ee7bec-2c85-48bd-80a3-1a427cb2cc0b admin admin] PATH_INFO: `/v3/projects/5e984423596a4cffa83e0b537e72ac88` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:22.430542 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ee7bec-2c85-48bd-80a3-1a427cb2cc0b admin admin] RBAC: Authorizing `identity:delete_project(project_id=5e984423596a4cffa83e0b537e72ac88)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:59:22.433088 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ee7bec-2c85-48bd-80a3-1a427cb2cc0b admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:59:22.500455 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-e9ee7bec-2c85-48bd-80a3-1a427cb2cc0b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5e984423596a4cffa83e0b537e72ac88', 'request_id': 'req-e9ee7bec-2c85-48bd-80a3-1a427cb2cc0b'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:22.503591 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:59:22.508618 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-e9ee7bec-2c85-48bd-80a3-1a427cb2cc0b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5e984423596a4cffa83e0b537e72ac88', 'request_id': 'req-e9ee7bec-2c85-48bd-80a3-1a427cb2cc0b'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:22.518548 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-e9ee7bec-2c85-48bd-80a3-1a427cb2cc0b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5e984423596a4cffa83e0b537e72ac88', 'request_id': 'req-e9ee7bec-2c85-48bd-80a3-1a427cb2cc0b'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:22.534548 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-e9ee7bec-2c85-48bd-80a3-1a427cb2cc0b admin admin] The token cache is being invalidate because project 5e984423596a4cffa83e0b537e72ac88 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81341) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:59:22.534835 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-e9ee7bec-2c85-48bd-80a3-1a427cb2cc0b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:22.535507 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-e9ee7bec-2c85-48bd-80a3-1a427cb2cc0b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:22.536995 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 969/1941] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Sep 1 14:59:22 2022] DELETE /identity/v3/projects/5e984423596a4cffa83e0b537e72ac88 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:59:22.542260 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80ef589-f8cd-40fc-9595-21ce9645063b None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:22.542574 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80ef589-f8cd-40fc-9595-21ce9645063b None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:22.542850 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80ef589-f8cd-40fc-9595-21ce9645063b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:22.563680 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-b80ef589-f8cd-40fc-9595-21ce9645063b None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:59:22.565308 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-b80ef589-f8cd-40fc-9595-21ce9645063b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:59:22.623225 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:59:22.642104 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 973/1942] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 14:59:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:59:22.643644 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0cacc1-91ff-4f9f-b949-a7e130f6f936 None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:22.644172 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0cacc1-91ff-4f9f-b949-a7e130f6f936 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:22.644172 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0cacc1-91ff-4f9f-b949-a7e130f6f936 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:22.666822 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-da0cacc1-91ff-4f9f-b949-a7e130f6f936 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:59:22.667650 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-da0cacc1-91ff-4f9f-b949-a7e130f6f936 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:59:22.677690 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 970/1943] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 14:59:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:59:22.682326 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80ef589-f8cd-40fc-9595-21ce9645063b None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:59:22.696350 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80ef589-f8cd-40fc-9595-21ce9645063b None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:59:22.701150 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd962278-d06d-4736-970b-4ce2cb6e05a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:59:22.701867 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd962278-d06d-4736-970b-4ce2cb6e05a8 admin admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:22.702258 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd962278-d06d-4736-970b-4ce2cb6e05a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:22.702646 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd962278-d06d-4736-970b-4ce2cb6e05a8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:22.703402 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd962278-d06d-4736-970b-4ce2cb6e05a8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:59:22.703804 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd962278-d06d-4736-970b-4ce2cb6e05a8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:59:22.705615 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd962278-d06d-4736-970b-4ce2cb6e05a8 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:59:22.727262 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 974/1944] 10.222.0.33 () {58 vars in 1153 bytes} [Thu Sep 1 14:59:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:59:22.731629 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0cacc1-91ff-4f9f-b949-a7e130f6f936 None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:59:22.743038 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0cacc1-91ff-4f9f-b949-a7e130f6f936 None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:59:22.748664 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc0985d-f6d4-45a8-8370-ea2b99cee66d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:59:22.749395 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc0985d-f6d4-45a8-8370-ea2b99cee66d admin admin] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:22.749883 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc0985d-f6d4-45a8-8370-ea2b99cee66d admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:22.750263 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc0985d-f6d4-45a8-8370-ea2b99cee66d admin admin] PATH_INFO: `/v3/projects` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:22.751226 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc0985d-f6d4-45a8-8370-ea2b99cee66d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:59:22.753474 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc0985d-f6d4-45a8-8370-ea2b99cee66d admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:59:22.785965 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 971/1945] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Sep 1 14:59:22 2022] POST /identity/v3/projects => generated 379 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:59:22.790609 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd962278-d06d-4736-970b-4ce2cb6e05a8 admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:59:22.800871 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd962278-d06d-4736-970b-4ce2cb6e05a8 admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:59:22.863917 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde41aa1-9137-438e-9cf8-1630160c5a8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:59:22.863917 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde41aa1-9137-438e-9cf8-1630160c5a8e admin admin] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:22.863917 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde41aa1-9137-438e-9cf8-1630160c5a8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:22.866475 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde41aa1-9137-438e-9cf8-1630160c5a8e admin admin] PATH_INFO: `/v3/users` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:22.867664 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde41aa1-9137-438e-9cf8-1630160c5a8e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:59:22.870040 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde41aa1-9137-438e-9cf8-1630160c5a8e admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:59:22.898629 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 975/1946] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Sep 1 14:59:22 2022] POST /identity/v3/users => generated 290 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:59:22.904330 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc0985d-f6d4-45a8-8370-ea2b99cee66d admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:59:22.917246 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc0985d-f6d4-45a8-8370-ea2b99cee66d admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:59:22.922576 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f56af7-eba2-4e9e-b51f-f9ff8ea389f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:59:22.936026 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f56af7-eba2-4e9e-b51f-f9ff8ea389f5 admin admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:22.937327 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f56af7-eba2-4e9e-b51f-f9ff8ea389f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:22.943471 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f56af7-eba2-4e9e-b51f-f9ff8ea389f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:22.944513 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f56af7-eba2-4e9e-b51f-f9ff8ea389f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:59:22.948118 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f56af7-eba2-4e9e-b51f-f9ff8ea389f5 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:59:22.980383 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 972/1947] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 1 14:59:22 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:59:22.981725 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde41aa1-9137-438e-9cf8-1630160c5a8e admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:59:22.999416 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde41aa1-9137-438e-9cf8-1630160c5a8e admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:59:23.005952 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15db1783-f05f-4cc6-8fc8-adbd89e4b86c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:59:23.007402 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15db1783-f05f-4cc6-8fc8-adbd89e4b86c admin admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:23.008085 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15db1783-f05f-4cc6-8fc8-adbd89e4b86c admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:23.009113 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15db1783-f05f-4cc6-8fc8-adbd89e4b86c admin admin] PATH_INFO: `/v3/roles` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:23.010567 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15db1783-f05f-4cc6-8fc8-adbd89e4b86c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:59:23.018464 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15db1783-f05f-4cc6-8fc8-adbd89e4b86c admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:59:23.035521 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 976/1948] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 1 14:59:22 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:59:23.040730 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f56af7-eba2-4e9e-b51f-f9ff8ea389f5 admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:59:23.052187 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f56af7-eba2-4e9e-b51f-f9ff8ea389f5 admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:59:23.057806 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39666de-5ac7-40ab-b2c6-9d37b79b5894 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:59:23.058990 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39666de-5ac7-40ab-b2c6-9d37b79b5894 admin admin] REQUEST_METHOD: `PUT` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:23.059516 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39666de-5ac7-40ab-b2c6-9d37b79b5894 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:23.060078 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39666de-5ac7-40ab-b2c6-9d37b79b5894 admin admin] PATH_INFO: `/v3/projects/095369e27f2c4bf2bbf1a39dff3957b7/users/e86eb68a85b44d479fc55d1d5f9f738b/roles/5d1feef0e13d4874928bd86258c4e3b2` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:23.080125 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39666de-5ac7-40ab-b2c6-9d37b79b5894 admin admin] RBAC: Authorizing `identity:create_grant(project_id=095369e27f2c4bf2bbf1a39dff3957b7, user_id=e86eb68a85b44d479fc55d1d5f9f738b, role_id=5d1feef0e13d4874928bd86258c4e3b2)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:59:23.083723 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39666de-5ac7-40ab-b2c6-9d37b79b5894 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:59:23.108129 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 973/1949] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 1 14:59:23 2022] PUT /identity/v3/projects/095369e27f2c4bf2bbf1a39dff3957b7/users/e86eb68a85b44d479fc55d1d5f9f738b/roles/5d1feef0e13d4874928bd86258c4e3b2 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:59:23.128406 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91204887-d062-4799-a008-3ee1738ac662 None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:23.128701 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91204887-d062-4799-a008-3ee1738ac662 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:23.128974 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91204887-d062-4799-a008-3ee1738ac662 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:23.161826 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-91204887-d062-4799-a008-3ee1738ac662 None None] MFA Rules not processed for user `e86eb68a85b44d479fc55d1d5f9f738b`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:59:23.163131 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-91204887-d062-4799-a008-3ee1738ac662 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:59:23.227812 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 977/1950] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 14:59:23 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:59:23.232599 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39666de-5ac7-40ab-b2c6-9d37b79b5894 admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:59:23.300580 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39666de-5ac7-40ab-b2c6-9d37b79b5894 admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:59:23.306962 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d17451-4715-4470-b716-29876a93673e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:59:23.307603 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d17451-4715-4470-b716-29876a93673e admin admin] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:23.307888 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d17451-4715-4470-b716-29876a93673e admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:23.308195 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d17451-4715-4470-b716-29876a93673e admin admin] PATH_INFO: `/v3/projects` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:23.308951 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d17451-4715-4470-b716-29876a93673e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:59:23.311241 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d17451-4715-4470-b716-29876a93673e admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:59:23.340269 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 974/1951] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Sep 1 14:59:23 2022] POST /identity/v3/projects => generated 381 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:59:23.344523 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91204887-d062-4799-a008-3ee1738ac662 None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:59:23.355966 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91204887-d062-4799-a008-3ee1738ac662 None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:59:23.411357 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:59:23.420494 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1fb70e-d1c4-43a1-a26d-0757575662b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:59:23.421194 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1fb70e-d1c4-43a1-a26d-0757575662b9 admin admin] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:23.421590 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1fb70e-d1c4-43a1-a26d-0757575662b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:23.421840 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1fb70e-d1c4-43a1-a26d-0757575662b9 admin admin] PATH_INFO: `/v3/users` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:23.422691 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1fb70e-d1c4-43a1-a26d-0757575662b9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:59:23.425883 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1fb70e-d1c4-43a1-a26d-0757575662b9 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:59:23.450498 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 978/1952] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Sep 1 14:59:23 2022] POST /identity/v3/users => generated 297 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:59:23.454736 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d17451-4715-4470-b716-29876a93673e admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:59:23.465929 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d17451-4715-4470-b716-29876a93673e admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:59:23.470934 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b26f9a-c100-4304-a6eb-a50c87640e48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:59:23.471560 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b26f9a-c100-4304-a6eb-a50c87640e48 admin admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:23.471867 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b26f9a-c100-4304-a6eb-a50c87640e48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:23.472142 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b26f9a-c100-4304-a6eb-a50c87640e48 admin admin] PATH_INFO: `/v3/roles` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:23.472785 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b26f9a-c100-4304-a6eb-a50c87640e48 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:59:23.474479 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b26f9a-c100-4304-a6eb-a50c87640e48 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:59:23.486913 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 975/1953] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 1 14:59:23 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:59:23.491791 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1fb70e-d1c4-43a1-a26d-0757575662b9 admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:59:23.503802 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1fb70e-d1c4-43a1-a26d-0757575662b9 admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:59:23.508096 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec22b60-945a-4495-9bb9-6189497f9647 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:59:23.508836 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec22b60-945a-4495-9bb9-6189497f9647 admin admin] REQUEST_METHOD: `PUT` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:23.509191 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec22b60-945a-4495-9bb9-6189497f9647 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:23.509566 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec22b60-945a-4495-9bb9-6189497f9647 admin admin] PATH_INFO: `/v3/projects/d7acaee8811a4cdba43a7fb3e9610b26/users/66c59917198a4a8fbfc20dc9b9b95514/roles/2a2adb769e55476b82e5014897e0a9c0` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:23.530028 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec22b60-945a-4495-9bb9-6189497f9647 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d7acaee8811a4cdba43a7fb3e9610b26, user_id=66c59917198a4a8fbfc20dc9b9b95514, role_id=2a2adb769e55476b82e5014897e0a9c0)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:59:23.532876 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec22b60-945a-4495-9bb9-6189497f9647 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:59:23.548229 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 979/1954] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 1 14:59:23 2022] PUT /identity/v3/projects/d7acaee8811a4cdba43a7fb3e9610b26/users/66c59917198a4a8fbfc20dc9b9b95514/roles/2a2adb769e55476b82e5014897e0a9c0 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:59:23.552836 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b26f9a-c100-4304-a6eb-a50c87640e48 admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:59:23.564638 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b26f9a-c100-4304-a6eb-a50c87640e48 admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:59:23.617485 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:59:23.632243 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67189688-daad-4a34-847a-c4d2130bcfd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:59:23.633173 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67189688-daad-4a34-847a-c4d2130bcfd6 admin admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:23.633454 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67189688-daad-4a34-847a-c4d2130bcfd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:23.633732 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67189688-daad-4a34-847a-c4d2130bcfd6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:23.634418 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67189688-daad-4a34-847a-c4d2130bcfd6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:59:23.636066 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67189688-daad-4a34-847a-c4d2130bcfd6 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:59:23.657414 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 976/1955] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 1 14:59:23 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:59:23.670626 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec22b60-945a-4495-9bb9-6189497f9647 admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:59:23.683140 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec22b60-945a-4495-9bb9-6189497f9647 admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:59:23.688297 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd667ac0-d267-4c49-9b5a-14ee68897925 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:59:23.689210 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd667ac0-d267-4c49-9b5a-14ee68897925 admin admin] REQUEST_METHOD: `PUT` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:23.689442 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd667ac0-d267-4c49-9b5a-14ee68897925 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:23.689715 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd667ac0-d267-4c49-9b5a-14ee68897925 admin admin] PATH_INFO: `/v3/projects/d7acaee8811a4cdba43a7fb3e9610b26/users/66c59917198a4a8fbfc20dc9b9b95514/roles/2a2adb769e55476b82e5014897e0a9c0` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:23.692241 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd667ac0-d267-4c49-9b5a-14ee68897925 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d7acaee8811a4cdba43a7fb3e9610b26, user_id=66c59917198a4a8fbfc20dc9b9b95514, role_id=2a2adb769e55476b82e5014897e0a9c0)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:59:23.694931 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd667ac0-d267-4c49-9b5a-14ee68897925 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:59:23.722938 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 980/1956] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 1 14:59:23 2022] PUT /identity/v3/projects/d7acaee8811a4cdba43a7fb3e9610b26/users/66c59917198a4a8fbfc20dc9b9b95514/roles/2a2adb769e55476b82e5014897e0a9c0 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:59:23.724289 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f05fbc-5567-45ec-a44d-3b3f082556fd None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:23.724574 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f05fbc-5567-45ec-a44d-3b3f082556fd None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:23.725877 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f05fbc-5567-45ec-a44d-3b3f082556fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:23.766293 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-b6f05fbc-5567-45ec-a44d-3b3f082556fd None None] MFA Rules not processed for user `66c59917198a4a8fbfc20dc9b9b95514`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:59:23.766293 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-b6f05fbc-5567-45ec-a44d-3b3f082556fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:59:23.828934 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 977/1957] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 14:59:23 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:59:23.840457 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd667ac0-d267-4c49-9b5a-14ee68897925 admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:59:23.842212 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-fd667ac0-d267-4c49-9b5a-14ee68897925 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:59:23.901199 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd667ac0-d267-4c49-9b5a-14ee68897925 admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:59:23.904976 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614c9a7b-6132-46ca-9f27-2ccf816d3105 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3b81739e34e4e648f88b6d73fcb5e46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:59:23.905560 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614c9a7b-6132-46ca-9f27-2ccf816d3105 service cinder] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:23.905832 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614c9a7b-6132-46ca-9f27-2ccf816d3105 service cinder] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:23.906098 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614c9a7b-6132-46ca-9f27-2ccf816d3105 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:23.915052 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614c9a7b-6132-46ca-9f27-2ccf816d3105 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:59:23.916472 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614c9a7b-6132-46ca-9f27-2ccf816d3105 service cinder] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:59:23.925760 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 981/1958] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 14:59:23 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:59:24.105767 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f05fbc-5567-45ec-a44d-3b3f082556fd None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:59:24.118208 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f05fbc-5567-45ec-a44d-3b3f082556fd None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:59:24.122985 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b48631b-2e18-494b-ae73-dc25951fffaf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3b81739e34e4e648f88b6d73fcb5e46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:59:24.123660 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b48631b-2e18-494b-ae73-dc25951fffaf service cinder] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:24.124008 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b48631b-2e18-494b-ae73-dc25951fffaf service cinder] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:24.124416 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b48631b-2e18-494b-ae73-dc25951fffaf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:24.211029 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b48631b-2e18-494b-ae73-dc25951fffaf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:59:24.215534 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b48631b-2e18-494b-ae73-dc25951fffaf service cinder] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:59:24.225792 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 978/1959] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 14:59:24 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:59:29.733485 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614c9a7b-6132-46ca-9f27-2ccf816d3105 service cinder] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:59:29.735162 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-614c9a7b-6132-46ca-9f27-2ccf816d3105 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:59:29.808780 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614c9a7b-6132-46ca-9f27-2ccf816d3105 service cinder] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:59:29.814083 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66763c8d-b784-4195-a079-542640f047b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:59:29.814934 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66763c8d-b784-4195-a079-542640f047b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:29.815991 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66763c8d-b784-4195-a079-542640f047b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:29.816432 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66763c8d-b784-4195-a079-542640f047b4 admin admin] PATH_INFO: `/v3/users/54047e52131e433990fac55f95b0e728` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:29.817581 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66763c8d-b784-4195-a079-542640f047b4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=54047e52131e433990fac55f95b0e728)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:59:29.820005 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66763c8d-b784-4195-a079-542640f047b4 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:59:29.885175 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-66763c8d-b784-4195-a079-542640f047b4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '54047e52131e433990fac55f95b0e728'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:29.889677 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=86,keystone:INSERT=6,keystone:DELETE=8 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:59:29.894088 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-66763c8d-b784-4195-a079-542640f047b4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '54047e52131e433990fac55f95b0e728'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:29.895196 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-66763c8d-b784-4195-a079-542640f047b4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '54047e52131e433990fac55f95b0e728'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:29.909486 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-66763c8d-b784-4195-a079-542640f047b4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '54047e52131e433990fac55f95b0e728'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:29.910214 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-66763c8d-b784-4195-a079-542640f047b4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '54047e52131e433990fac55f95b0e728'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:29.927257 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 982/1960] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Sep 1 14:59:29 2022] DELETE /identity/v3/users/54047e52131e433990fac55f95b0e728 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:59:29.933483 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111536c9-5c26-4770-b437-2bdf4eff08ee None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:29.934648 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111536c9-5c26-4770-b437-2bdf4eff08ee None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:29.934941 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111536c9-5c26-4770-b437-2bdf4eff08ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:29.956716 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-111536c9-5c26-4770-b437-2bdf4eff08ee None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:59:29.958064 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-111536c9-5c26-4770-b437-2bdf4eff08ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:59:30.028734 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 979/1961] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 14:59:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:59:30.038593 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66763c8d-b784-4195-a079-542640f047b4 admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:59:30.040034 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-66763c8d-b784-4195-a079-542640f047b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:59:30.098597 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66763c8d-b784-4195-a079-542640f047b4 admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:59:30.102225 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc017bb-d36e-4288-8fbd-b0c8b89ac618 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2790149639aa418bbc4e4982e93eda44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:59:30.102861 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc017bb-d36e-4288-8fbd-b0c8b89ac618 service neutron] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:30.103139 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc017bb-d36e-4288-8fbd-b0c8b89ac618 service neutron] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:30.103410 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc017bb-d36e-4288-8fbd-b0c8b89ac618 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:30.112810 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc017bb-d36e-4288-8fbd-b0c8b89ac618 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:59:30.114257 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc017bb-d36e-4288-8fbd-b0c8b89ac618 service neutron] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:59:30.123578 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 983/1962] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 14:59:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:59:30.387888 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111536c9-5c26-4770-b437-2bdf4eff08ee None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:59:30.389303 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-111536c9-5c26-4770-b437-2bdf4eff08ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:59:30.401305 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111536c9-5c26-4770-b437-2bdf4eff08ee None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:59:30.406693 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc11e16-aa12-477e-82f9-2eb94fd1275b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:59:30.407355 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc11e16-aa12-477e-82f9-2eb94fd1275b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:30.407634 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc11e16-aa12-477e-82f9-2eb94fd1275b admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:30.407908 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc11e16-aa12-477e-82f9-2eb94fd1275b admin admin] PATH_INFO: `/v3/projects/8f96f009ba564b7993f50e5ca10a2cac` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:30.409918 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc11e16-aa12-477e-82f9-2eb94fd1275b admin admin] RBAC: Authorizing `identity:delete_project(project_id=8f96f009ba564b7993f50e5ca10a2cac)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:59:30.411878 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc11e16-aa12-477e-82f9-2eb94fd1275b admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:59:30.418509 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:59:30.458747 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-1dc11e16-aa12-477e-82f9-2eb94fd1275b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8f96f009ba564b7993f50e5ca10a2cac', 'request_id': 'req-1dc11e16-aa12-477e-82f9-2eb94fd1275b'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:30.464841 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-1dc11e16-aa12-477e-82f9-2eb94fd1275b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8f96f009ba564b7993f50e5ca10a2cac', 'request_id': 'req-1dc11e16-aa12-477e-82f9-2eb94fd1275b'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:30.471961 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-1dc11e16-aa12-477e-82f9-2eb94fd1275b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8f96f009ba564b7993f50e5ca10a2cac', 'request_id': 'req-1dc11e16-aa12-477e-82f9-2eb94fd1275b'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:30.483263 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-1dc11e16-aa12-477e-82f9-2eb94fd1275b admin admin] The token cache is being invalidate because project 8f96f009ba564b7993f50e5ca10a2cac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81341) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:59:30.483552 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-1dc11e16-aa12-477e-82f9-2eb94fd1275b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:30.484195 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-1dc11e16-aa12-477e-82f9-2eb94fd1275b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:30.485679 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 980/1963] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Sep 1 14:59:30 2022] DELETE /identity/v3/projects/8f96f009ba564b7993f50e5ca10a2cac => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 14:59:30.490380 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc017bb-d36e-4288-8fbd-b0c8b89ac618 service neutron] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:59:30.500069 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc017bb-d36e-4288-8fbd-b0c8b89ac618 service neutron] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:59:30.556662 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9752cd4a-6221-4a85-af85-5a31a1a3f727 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:59:30.557324 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9752cd4a-6221-4a85-af85-5a31a1a3f727 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:30.557604 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9752cd4a-6221-4a85-af85-5a31a1a3f727 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:30.557878 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9752cd4a-6221-4a85-af85-5a31a1a3f727 admin admin] PATH_INFO: `/v3/users/1435b67048d1400597001f34ce44a316` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:30.559037 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9752cd4a-6221-4a85-af85-5a31a1a3f727 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1435b67048d1400597001f34ce44a316)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:59:30.560956 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9752cd4a-6221-4a85-af85-5a31a1a3f727 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:59:30.604655 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-9752cd4a-6221-4a85-af85-5a31a1a3f727 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1435b67048d1400597001f34ce44a316'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:30.612926 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-9752cd4a-6221-4a85-af85-5a31a1a3f727 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1435b67048d1400597001f34ce44a316'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:30.613543 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-9752cd4a-6221-4a85-af85-5a31a1a3f727 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1435b67048d1400597001f34ce44a316'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:30.625003 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-9752cd4a-6221-4a85-af85-5a31a1a3f727 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1435b67048d1400597001f34ce44a316'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:30.625616 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-9752cd4a-6221-4a85-af85-5a31a1a3f727 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1435b67048d1400597001f34ce44a316'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:30.627577 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:59:30.639354 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 984/1964] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Sep 1 14:59:30 2022] DELETE /identity/v3/users/1435b67048d1400597001f34ce44a316 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:59:30.906129 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc11e16-aa12-477e-82f9-2eb94fd1275b admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:59:30.907723 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-1dc11e16-aa12-477e-82f9-2eb94fd1275b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:59:30.925064 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc11e16-aa12-477e-82f9-2eb94fd1275b admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:59:30.984300 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b52f58-8557-4bb1-9718-646786a93b3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:59:30.984937 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b52f58-8557-4bb1-9718-646786a93b3f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:30.985215 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b52f58-8557-4bb1-9718-646786a93b3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:30.985484 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b52f58-8557-4bb1-9718-646786a93b3f admin admin] PATH_INFO: `/v3/projects/0910b87ac4b3494aaf2a308e8a5d7bd7` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:30.986643 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b52f58-8557-4bb1-9718-646786a93b3f admin admin] RBAC: Authorizing `identity:delete_project(project_id=0910b87ac4b3494aaf2a308e8a5d7bd7)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:59:30.988809 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b52f58-8557-4bb1-9718-646786a93b3f admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:59:31.048124 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-35b52f58-8557-4bb1-9718-646786a93b3f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0910b87ac4b3494aaf2a308e8a5d7bd7', 'request_id': 'req-35b52f58-8557-4bb1-9718-646786a93b3f'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:31.054882 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-35b52f58-8557-4bb1-9718-646786a93b3f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0910b87ac4b3494aaf2a308e8a5d7bd7', 'request_id': 'req-35b52f58-8557-4bb1-9718-646786a93b3f'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:31.063447 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-35b52f58-8557-4bb1-9718-646786a93b3f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0910b87ac4b3494aaf2a308e8a5d7bd7', 'request_id': 'req-35b52f58-8557-4bb1-9718-646786a93b3f'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:31.073840 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-35b52f58-8557-4bb1-9718-646786a93b3f admin admin] The token cache is being invalidate because project 0910b87ac4b3494aaf2a308e8a5d7bd7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81341) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:59:31.074128 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-35b52f58-8557-4bb1-9718-646786a93b3f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:31.074868 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-35b52f58-8557-4bb1-9718-646786a93b3f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:31.076367 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 981/1965] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Sep 1 14:59:30 2022] DELETE /identity/v3/projects/0910b87ac4b3494aaf2a308e8a5d7bd7 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:59:31.083172 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c23c07-839e-424b-9368-963795b2121b None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:31.083659 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c23c07-839e-424b-9368-963795b2121b None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:31.083758 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c23c07-839e-424b-9368-963795b2121b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:31.104180 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-e8c23c07-839e-424b-9368-963795b2121b None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:59:31.105391 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-e8c23c07-839e-424b-9368-963795b2121b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:59:31.163161 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 985/1966] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 14:59:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:59:31.172196 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2771822-df6a-4cd7-be10-4bb31729b6db None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:31.172464 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2771822-df6a-4cd7-be10-4bb31729b6db None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:31.172734 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2771822-df6a-4cd7-be10-4bb31729b6db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:31.192776 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-e2771822-df6a-4cd7-be10-4bb31729b6db None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:59:31.193962 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-e2771822-df6a-4cd7-be10-4bb31729b6db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:59:31.201607 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 982/1967] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 14:59:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:59:31.205754 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c23c07-839e-424b-9368-963795b2121b None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:59:31.216039 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c23c07-839e-424b-9368-963795b2121b None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:59:31.220656 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d08c2d-65a9-4c49-ac5b-e9f29a9c814f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:59:31.221249 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d08c2d-65a9-4c49-ac5b-e9f29a9c814f admin admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:31.221531 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d08c2d-65a9-4c49-ac5b-e9f29a9c814f admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:31.221803 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d08c2d-65a9-4c49-ac5b-e9f29a9c814f admin admin] PATH_INFO: `/v3/domains` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:31.222490 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d08c2d-65a9-4c49-ac5b-e9f29a9c814f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:59:31.222794 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d08c2d-65a9-4c49-ac5b-e9f29a9c814f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:59:31.224378 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d08c2d-65a9-4c49-ac5b-e9f29a9c814f admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:59:31.235790 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 986/1968] 10.222.0.33 () {58 vars in 1153 bytes} [Thu Sep 1 14:59:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:59:31.239948 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2771822-df6a-4cd7-be10-4bb31729b6db None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:59:31.248963 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2771822-df6a-4cd7-be10-4bb31729b6db None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:59:31.253953 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa74cb5-dc4d-45a5-a2eb-13bfdedaf0fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:59:31.254555 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa74cb5-dc4d-45a5-a2eb-13bfdedaf0fa admin admin] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:31.254831 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa74cb5-dc4d-45a5-a2eb-13bfdedaf0fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:31.255094 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa74cb5-dc4d-45a5-a2eb-13bfdedaf0fa admin admin] PATH_INFO: `/v3/projects` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:31.255707 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa74cb5-dc4d-45a5-a2eb-13bfdedaf0fa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:59:31.257613 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa74cb5-dc4d-45a5-a2eb-13bfdedaf0fa admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:59:31.273228 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:59:31.286484 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 983/1969] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Sep 1 14:59:31 2022] POST /identity/v3/projects => generated 373 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:59:31.290441 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d08c2d-65a9-4c49-ac5b-e9f29a9c814f admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:59:31.300579 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d08c2d-65a9-4c49-ac5b-e9f29a9c814f admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:59:31.355990 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59f0521-0b55-4472-ac31-83761474c83a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:59:31.356594 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59f0521-0b55-4472-ac31-83761474c83a admin admin] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:31.356879 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59f0521-0b55-4472-ac31-83761474c83a admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:31.357151 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59f0521-0b55-4472-ac31-83761474c83a admin admin] PATH_INFO: `/v3/users` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:31.357786 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59f0521-0b55-4472-ac31-83761474c83a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:59:31.359743 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59f0521-0b55-4472-ac31-83761474c83a admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:59:31.380958 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 987/1970] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Sep 1 14:59:31 2022] POST /identity/v3/users => generated 287 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:59:31.385088 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa74cb5-dc4d-45a5-a2eb-13bfdedaf0fa admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:59:31.396235 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa74cb5-dc4d-45a5-a2eb-13bfdedaf0fa admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:59:31.400984 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd5a4a8-45ef-4f0c-bcea-a7689da6df86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:59:31.401581 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd5a4a8-45ef-4f0c-bcea-a7689da6df86 admin admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:31.401861 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd5a4a8-45ef-4f0c-bcea-a7689da6df86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:31.402211 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd5a4a8-45ef-4f0c-bcea-a7689da6df86 admin admin] PATH_INFO: `/v3/roles` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:31.402904 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd5a4a8-45ef-4f0c-bcea-a7689da6df86 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:59:31.404512 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd5a4a8-45ef-4f0c-bcea-a7689da6df86 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:59:31.414821 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 984/1971] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 1 14:59:31 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:59:31.421142 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59f0521-0b55-4472-ac31-83761474c83a admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:59:31.431609 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59f0521-0b55-4472-ac31-83761474c83a admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:59:31.436220 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9eb2b44-7d98-426c-8779-8b1b675d201d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:59:31.436841 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9eb2b44-7d98-426c-8779-8b1b675d201d admin admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:31.437121 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9eb2b44-7d98-426c-8779-8b1b675d201d admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:31.437392 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9eb2b44-7d98-426c-8779-8b1b675d201d admin admin] PATH_INFO: `/v3/roles` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:31.438063 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9eb2b44-7d98-426c-8779-8b1b675d201d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:59:31.439726 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9eb2b44-7d98-426c-8779-8b1b675d201d admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:59:31.450780 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 988/1972] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 1 14:59:31 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:59:31.454806 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd5a4a8-45ef-4f0c-bcea-a7689da6df86 admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:59:31.465304 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd5a4a8-45ef-4f0c-bcea-a7689da6df86 admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:59:31.469939 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c58e69-9bfc-4145-8856-3fd847b4de83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:59:31.470615 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c58e69-9bfc-4145-8856-3fd847b4de83 admin admin] REQUEST_METHOD: `PUT` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:31.470900 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c58e69-9bfc-4145-8856-3fd847b4de83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:31.471181 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c58e69-9bfc-4145-8856-3fd847b4de83 admin admin] PATH_INFO: `/v3/projects/dbbbcbbddfb7460dac1fc015a856c20a/users/f6e4b8cfe1c3459694086ea7513196d0/roles/5d1feef0e13d4874928bd86258c4e3b2` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:31.485265 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c58e69-9bfc-4145-8856-3fd847b4de83 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dbbbcbbddfb7460dac1fc015a856c20a, user_id=f6e4b8cfe1c3459694086ea7513196d0, role_id=5d1feef0e13d4874928bd86258c4e3b2)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:59:31.488007 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c58e69-9bfc-4145-8856-3fd847b4de83 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:59:31.500329 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 985/1973] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 1 14:59:31 2022] PUT /identity/v3/projects/dbbbcbbddfb7460dac1fc015a856c20a/users/f6e4b8cfe1c3459694086ea7513196d0/roles/5d1feef0e13d4874928bd86258c4e3b2 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:59:31.512827 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd53b96-3c2b-4b65-861e-8828f4f6818a None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:31.513111 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd53b96-3c2b-4b65-861e-8828f4f6818a None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:31.513378 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd53b96-3c2b-4b65-861e-8828f4f6818a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:31.541107 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-4cd53b96-3c2b-4b65-861e-8828f4f6818a None None] MFA Rules not processed for user `f6e4b8cfe1c3459694086ea7513196d0`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:59:31.542355 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-4cd53b96-3c2b-4b65-861e-8828f4f6818a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:59:31.587921 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:59:31.609055 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 989/1974] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 14:59:31 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:59:31.614026 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c58e69-9bfc-4145-8856-3fd847b4de83 admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:59:31.678226 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c58e69-9bfc-4145-8856-3fd847b4de83 admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:59:31.682833 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b1d582-ba68-4c11-b4d3-5a608b65fa12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:59:31.683432 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b1d582-ba68-4c11-b4d3-5a608b65fa12 admin admin] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:31.683708 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b1d582-ba68-4c11-b4d3-5a608b65fa12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:31.683977 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b1d582-ba68-4c11-b4d3-5a608b65fa12 admin admin] PATH_INFO: `/v3/projects` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:31.684601 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b1d582-ba68-4c11-b4d3-5a608b65fa12 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:59:31.686587 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b1d582-ba68-4c11-b4d3-5a608b65fa12 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:59:31.712484 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 986/1975] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Sep 1 14:59:31 2022] POST /identity/v3/projects => generated 375 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:59:31.716570 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd53b96-3c2b-4b65-861e-8828f4f6818a None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:59:31.727027 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd53b96-3c2b-4b65-861e-8828f4f6818a None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:59:31.784634 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c91319-40e0-46ed-9350-967e026d9737 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:59:31.785393 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c91319-40e0-46ed-9350-967e026d9737 admin admin] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:31.785670 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c91319-40e0-46ed-9350-967e026d9737 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:31.785930 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c91319-40e0-46ed-9350-967e026d9737 admin admin] PATH_INFO: `/v3/users` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:31.786580 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c91319-40e0-46ed-9350-967e026d9737 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:59:31.788499 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c91319-40e0-46ed-9350-967e026d9737 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:59:31.813890 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 990/1976] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Sep 1 14:59:31 2022] POST /identity/v3/users => generated 294 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:59:31.815781 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b1d582-ba68-4c11-b4d3-5a608b65fa12 admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:59:31.825833 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b1d582-ba68-4c11-b4d3-5a608b65fa12 admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:59:31.830365 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a49724a-b905-4e8d-ad2d-4e75ee142eff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:59:31.830956 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a49724a-b905-4e8d-ad2d-4e75ee142eff admin admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:31.831231 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a49724a-b905-4e8d-ad2d-4e75ee142eff admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:31.831497 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a49724a-b905-4e8d-ad2d-4e75ee142eff admin admin] PATH_INFO: `/v3/roles` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:31.832126 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a49724a-b905-4e8d-ad2d-4e75ee142eff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:59:31.833826 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a49724a-b905-4e8d-ad2d-4e75ee142eff admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:59:31.844124 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 987/1977] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 1 14:59:31 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:59:31.848261 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c91319-40e0-46ed-9350-967e026d9737 admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:59:31.858749 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c91319-40e0-46ed-9350-967e026d9737 admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:59:31.863875 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006c2e0f-af6d-4f7f-879a-bab3cbeae8df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:59:31.864518 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006c2e0f-af6d-4f7f-879a-bab3cbeae8df admin admin] REQUEST_METHOD: `PUT` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:31.864802 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006c2e0f-af6d-4f7f-879a-bab3cbeae8df admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:31.865075 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006c2e0f-af6d-4f7f-879a-bab3cbeae8df admin admin] PATH_INFO: `/v3/projects/dd561c731f4b4f83a5eb215f1693cbcd/users/511adfae453140e28cc11116df8d1b2e/roles/2a2adb769e55476b82e5014897e0a9c0` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:31.880186 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006c2e0f-af6d-4f7f-879a-bab3cbeae8df admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd561c731f4b4f83a5eb215f1693cbcd, user_id=511adfae453140e28cc11116df8d1b2e, role_id=2a2adb769e55476b82e5014897e0a9c0)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:59:31.882932 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006c2e0f-af6d-4f7f-879a-bab3cbeae8df admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:59:31.894939 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 991/1978] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 1 14:59:31 2022] PUT /identity/v3/projects/dd561c731f4b4f83a5eb215f1693cbcd/users/511adfae453140e28cc11116df8d1b2e/roles/2a2adb769e55476b82e5014897e0a9c0 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:59:31.898762 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a49724a-b905-4e8d-ad2d-4e75ee142eff admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:59:31.909728 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a49724a-b905-4e8d-ad2d-4e75ee142eff admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:59:31.964849 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539b4392-b1b6-41a2-a44a-cf3cbb255952 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:59:31.965389 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539b4392-b1b6-41a2-a44a-cf3cbb255952 admin admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:31.965671 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539b4392-b1b6-41a2-a44a-cf3cbb255952 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:31.965946 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539b4392-b1b6-41a2-a44a-cf3cbb255952 admin admin] PATH_INFO: `/v3/roles` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:31.966617 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539b4392-b1b6-41a2-a44a-cf3cbb255952 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:59:31.968227 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539b4392-b1b6-41a2-a44a-cf3cbb255952 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:59:31.978298 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 988/1979] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 1 14:59:31 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:59:31.982486 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006c2e0f-af6d-4f7f-879a-bab3cbeae8df admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:59:31.993208 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006c2e0f-af6d-4f7f-879a-bab3cbeae8df admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:59:31.997661 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c58e83b-4097-41b7-974a-d7b2d8c88130 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:59:31.998334 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c58e83b-4097-41b7-974a-d7b2d8c88130 admin admin] REQUEST_METHOD: `PUT` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:31.998614 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c58e83b-4097-41b7-974a-d7b2d8c88130 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:31.998877 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c58e83b-4097-41b7-974a-d7b2d8c88130 admin admin] PATH_INFO: `/v3/projects/dd561c731f4b4f83a5eb215f1693cbcd/users/511adfae453140e28cc11116df8d1b2e/roles/2a2adb769e55476b82e5014897e0a9c0` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:32.000826 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c58e83b-4097-41b7-974a-d7b2d8c88130 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd561c731f4b4f83a5eb215f1693cbcd, user_id=511adfae453140e28cc11116df8d1b2e, role_id=2a2adb769e55476b82e5014897e0a9c0)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:59:32.003520 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c58e83b-4097-41b7-974a-d7b2d8c88130 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:59:32.013369 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 992/1980] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 1 14:59:31 2022] PUT /identity/v3/projects/dd561c731f4b4f83a5eb215f1693cbcd/users/511adfae453140e28cc11116df8d1b2e/roles/2a2adb769e55476b82e5014897e0a9c0 => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:59:32.024082 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b514c9-6c07-4f48-8b45-1f84a498283a None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:32.024570 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b514c9-6c07-4f48-8b45-1f84a498283a None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:32.024667 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b514c9-6c07-4f48-8b45-1f84a498283a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:32.054110 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-09b514c9-6c07-4f48-8b45-1f84a498283a None None] MFA Rules not processed for user `511adfae453140e28cc11116df8d1b2e`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:59:32.055363 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-09b514c9-6c07-4f48-8b45-1f84a498283a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:59:32.075686 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:59:32.138324 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 989/1981] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 14:59:32 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:59:32.149427 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c58e83b-4097-41b7-974a-d7b2d8c88130 admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:59:32.151266 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-7c58e83b-4097-41b7-974a-d7b2d8c88130 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:59:32.210008 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c58e83b-4097-41b7-974a-d7b2d8c88130 admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:59:32.213886 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd60c15-697b-4ab5-83e3-cea1446d835c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3b81739e34e4e648f88b6d73fcb5e46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:59:32.214647 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd60c15-697b-4ab5-83e3-cea1446d835c service cinder] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:32.215103 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd60c15-697b-4ab5-83e3-cea1446d835c service cinder] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:32.215387 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd60c15-697b-4ab5-83e3-cea1446d835c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:32.268682 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:59:32.273141 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd60c15-697b-4ab5-83e3-cea1446d835c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:59:32.275085 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd60c15-697b-4ab5-83e3-cea1446d835c service cinder] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:59:32.284447 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 993/1982] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 14:59:32 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:59:34.908035 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b514c9-6c07-4f48-8b45-1f84a498283a None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:59:34.976680 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b514c9-6c07-4f48-8b45-1f84a498283a None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:59:34.981353 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2c30cc-ae1c-4b42-932f-851ecfee446b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:59:34.982121 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2c30cc-ae1c-4b42-932f-851ecfee446b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:34.982442 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2c30cc-ae1c-4b42-932f-851ecfee446b admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:34.982717 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2c30cc-ae1c-4b42-932f-851ecfee446b admin admin] PATH_INFO: `/v3/users/f6e4b8cfe1c3459694086ea7513196d0` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:34.983949 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2c30cc-ae1c-4b42-932f-851ecfee446b admin admin] RBAC: Authorizing `identity:delete_user(user_id=f6e4b8cfe1c3459694086ea7513196d0)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:59:34.985902 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2c30cc-ae1c-4b42-932f-851ecfee446b admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:59:35.031164 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-ad2c30cc-ae1c-4b42-932f-851ecfee446b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f6e4b8cfe1c3459694086ea7513196d0'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:35.038764 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-ad2c30cc-ae1c-4b42-932f-851ecfee446b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f6e4b8cfe1c3459694086ea7513196d0'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:35.039379 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-ad2c30cc-ae1c-4b42-932f-851ecfee446b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f6e4b8cfe1c3459694086ea7513196d0'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:35.051900 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-ad2c30cc-ae1c-4b42-932f-851ecfee446b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f6e4b8cfe1c3459694086ea7513196d0'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:35.052548 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-ad2c30cc-ae1c-4b42-932f-851ecfee446b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f6e4b8cfe1c3459694086ea7513196d0'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:35.064825 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 990/1983] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Sep 1 14:59:34 2022] DELETE /identity/v3/users/f6e4b8cfe1c3459694086ea7513196d0 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:59:35.069291 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fa2f4f-adb3-41d6-a623-717bbec1196d None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:35.069762 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fa2f4f-adb3-41d6-a623-717bbec1196d None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:35.069898 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fa2f4f-adb3-41d6-a623-717bbec1196d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:35.088575 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-42fa2f4f-adb3-41d6-a623-717bbec1196d None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:59:35.089765 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-42fa2f4f-adb3-41d6-a623-717bbec1196d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:59:35.147519 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 994/1984] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 14:59:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:59:35.156995 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2c30cc-ae1c-4b42-932f-851ecfee446b admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:59:35.159225 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-ad2c30cc-ae1c-4b42-932f-851ecfee446b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:59:35.202299 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:59:35.223587 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2c30cc-ae1c-4b42-932f-851ecfee446b admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:59:35.228093 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bac88a5-6f0d-42e4-95c3-296c6d5d9e40 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2790149639aa418bbc4e4982e93eda44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:59:35.228801 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bac88a5-6f0d-42e4-95c3-296c6d5d9e40 service neutron] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:35.229081 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bac88a5-6f0d-42e4-95c3-296c6d5d9e40 service neutron] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:35.229511 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bac88a5-6f0d-42e4-95c3-296c6d5d9e40 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:35.247077 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bac88a5-6f0d-42e4-95c3-296c6d5d9e40 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:59:35.248784 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bac88a5-6f0d-42e4-95c3-296c6d5d9e40 service neutron] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:59:35.261339 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 991/1985] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 14:59:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:59:35.545556 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fa2f4f-adb3-41d6-a623-717bbec1196d None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:59:35.547137 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-42fa2f4f-adb3-41d6-a623-717bbec1196d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:59:35.567147 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fa2f4f-adb3-41d6-a623-717bbec1196d None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:59:35.572404 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a366c5d6-3ab1-4b04-a64d-0e1b19867d2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:59:35.573186 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a366c5d6-3ab1-4b04-a64d-0e1b19867d2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:35.573470 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a366c5d6-3ab1-4b04-a64d-0e1b19867d2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:35.573815 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a366c5d6-3ab1-4b04-a64d-0e1b19867d2c admin admin] PATH_INFO: `/v3/projects/dbbbcbbddfb7460dac1fc015a856c20a` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:35.575595 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a366c5d6-3ab1-4b04-a64d-0e1b19867d2c admin admin] RBAC: Authorizing `identity:delete_project(project_id=dbbbcbbddfb7460dac1fc015a856c20a)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:59:35.577589 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a366c5d6-3ab1-4b04-a64d-0e1b19867d2c admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:59:35.685978 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-a366c5d6-3ab1-4b04-a64d-0e1b19867d2c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dbbbcbbddfb7460dac1fc015a856c20a', 'request_id': 'req-a366c5d6-3ab1-4b04-a64d-0e1b19867d2c'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:35.693562 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-a366c5d6-3ab1-4b04-a64d-0e1b19867d2c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dbbbcbbddfb7460dac1fc015a856c20a', 'request_id': 'req-a366c5d6-3ab1-4b04-a64d-0e1b19867d2c'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:35.702231 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-a366c5d6-3ab1-4b04-a64d-0e1b19867d2c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dbbbcbbddfb7460dac1fc015a856c20a', 'request_id': 'req-a366c5d6-3ab1-4b04-a64d-0e1b19867d2c'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:35.714301 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-a366c5d6-3ab1-4b04-a64d-0e1b19867d2c admin admin] The token cache is being invalidate because project dbbbcbbddfb7460dac1fc015a856c20a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81342) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:59:35.714686 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-a366c5d6-3ab1-4b04-a64d-0e1b19867d2c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:35.715280 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-a366c5d6-3ab1-4b04-a64d-0e1b19867d2c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:35.716931 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 995/1986] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Sep 1 14:59:35 2022] DELETE /identity/v3/projects/dbbbcbbddfb7460dac1fc015a856c20a => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:59:35.721127 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bac88a5-6f0d-42e4-95c3-296c6d5d9e40 service neutron] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:59:35.731920 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bac88a5-6f0d-42e4-95c3-296c6d5d9e40 service neutron] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:59:35.802084 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3933e32a-0d89-45dd-aa92-f5eac14da9ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:59:35.803115 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3933e32a-0d89-45dd-aa92-f5eac14da9ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:35.803508 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3933e32a-0d89-45dd-aa92-f5eac14da9ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:35.804010 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3933e32a-0d89-45dd-aa92-f5eac14da9ec admin admin] PATH_INFO: `/v3/users/511adfae453140e28cc11116df8d1b2e` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:35.805527 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3933e32a-0d89-45dd-aa92-f5eac14da9ec admin admin] RBAC: Authorizing `identity:delete_user(user_id=511adfae453140e28cc11116df8d1b2e)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:59:35.807876 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3933e32a-0d89-45dd-aa92-f5eac14da9ec admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:59:35.882908 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-3933e32a-0d89-45dd-aa92-f5eac14da9ec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '511adfae453140e28cc11116df8d1b2e'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:35.894394 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-3933e32a-0d89-45dd-aa92-f5eac14da9ec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '511adfae453140e28cc11116df8d1b2e'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:35.895321 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-3933e32a-0d89-45dd-aa92-f5eac14da9ec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '511adfae453140e28cc11116df8d1b2e'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:35.912133 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-3933e32a-0d89-45dd-aa92-f5eac14da9ec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '511adfae453140e28cc11116df8d1b2e'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:35.912889 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-3933e32a-0d89-45dd-aa92-f5eac14da9ec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '511adfae453140e28cc11116df8d1b2e'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:35.930965 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 992/1987] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Sep 1 14:59:35 2022] DELETE /identity/v3/users/511adfae453140e28cc11116df8d1b2e => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:59:36.295829 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a366c5d6-3ab1-4b04-a64d-0e1b19867d2c admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:59:36.297742 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-a366c5d6-3ab1-4b04-a64d-0e1b19867d2c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:59:36.326113 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a366c5d6-3ab1-4b04-a64d-0e1b19867d2c admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:59:36.393815 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5be8d1-7f45-4cc0-bea5-72ccd3beedd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:59:36.394709 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5be8d1-7f45-4cc0-bea5-72ccd3beedd5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:36.395319 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5be8d1-7f45-4cc0-bea5-72ccd3beedd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:36.395426 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5be8d1-7f45-4cc0-bea5-72ccd3beedd5 admin admin] PATH_INFO: `/v3/projects/dd561c731f4b4f83a5eb215f1693cbcd` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:36.396618 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5be8d1-7f45-4cc0-bea5-72ccd3beedd5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dd561c731f4b4f83a5eb215f1693cbcd)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:59:36.398821 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5be8d1-7f45-4cc0-bea5-72ccd3beedd5 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:59:36.427234 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:59:36.465398 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-bf5be8d1-7f45-4cc0-bea5-72ccd3beedd5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dd561c731f4b4f83a5eb215f1693cbcd', 'request_id': 'req-bf5be8d1-7f45-4cc0-bea5-72ccd3beedd5'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:36.473020 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-bf5be8d1-7f45-4cc0-bea5-72ccd3beedd5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dd561c731f4b4f83a5eb215f1693cbcd', 'request_id': 'req-bf5be8d1-7f45-4cc0-bea5-72ccd3beedd5'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:36.482062 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-bf5be8d1-7f45-4cc0-bea5-72ccd3beedd5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dd561c731f4b4f83a5eb215f1693cbcd', 'request_id': 'req-bf5be8d1-7f45-4cc0-bea5-72ccd3beedd5'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:36.497529 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-bf5be8d1-7f45-4cc0-bea5-72ccd3beedd5 admin admin] The token cache is being invalidate because project dd561c731f4b4f83a5eb215f1693cbcd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81342) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:59:36.497823 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-bf5be8d1-7f45-4cc0-bea5-72ccd3beedd5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:36.498729 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-bf5be8d1-7f45-4cc0-bea5-72ccd3beedd5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:36.503893 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 996/1988] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Sep 1 14:59:36 2022] DELETE /identity/v3/projects/dd561c731f4b4f83a5eb215f1693cbcd => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:59:36.505852 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d55126c-52ec-481f-aa3f-ff8028fe24ca None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:36.506158 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d55126c-52ec-481f-aa3f-ff8028fe24ca None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:36.506456 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d55126c-52ec-481f-aa3f-ff8028fe24ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:36.528514 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-2d55126c-52ec-481f-aa3f-ff8028fe24ca None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:59:36.529734 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-2d55126c-52ec-481f-aa3f-ff8028fe24ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:59:36.618868 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:59:36.639226 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 993/1989] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 14:59:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 14:59:36.648421 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bad124-110c-49c1-8e10-22a35ca6410d None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:36.648713 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bad124-110c-49c1-8e10-22a35ca6410d None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:36.648972 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bad124-110c-49c1-8e10-22a35ca6410d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:36.689706 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-88bad124-110c-49c1-8e10-22a35ca6410d None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:59:36.692076 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-88bad124-110c-49c1-8e10-22a35ca6410d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:59:36.708644 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 997/1990] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 14:59:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:59:36.713352 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d55126c-52ec-481f-aa3f-ff8028fe24ca None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:59:36.724247 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bad124-110c-49c1-8e10-22a35ca6410d None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:59:36.726108 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-88bad124-110c-49c1-8e10-22a35ca6410d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:59:36.727109 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d55126c-52ec-481f-aa3f-ff8028fe24ca None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:59:36.741498 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6268876e-0711-4862-8cde-4c1f523cb5d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:59:36.742145 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6268876e-0711-4862-8cde-4c1f523cb5d0 admin admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:36.742458 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6268876e-0711-4862-8cde-4c1f523cb5d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:36.742727 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6268876e-0711-4862-8cde-4c1f523cb5d0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:36.743629 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6268876e-0711-4862-8cde-4c1f523cb5d0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81341) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:59:36.743934 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6268876e-0711-4862-8cde-4c1f523cb5d0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:59:36.744287 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bad124-110c-49c1-8e10-22a35ca6410d None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:59:36.745579 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6268876e-0711-4862-8cde-4c1f523cb5d0 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:59:36.751113 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3774004f-b460-4874-a1a5-43edcbdcb164 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:59:36.752680 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3774004f-b460-4874-a1a5-43edcbdcb164 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:36.752680 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3774004f-b460-4874-a1a5-43edcbdcb164 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:36.752680 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3774004f-b460-4874-a1a5-43edcbdcb164 admin admin] PATH_INFO: `/v3/users/e86eb68a85b44d479fc55d1d5f9f738b` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:36.754198 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3774004f-b460-4874-a1a5-43edcbdcb164 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e86eb68a85b44d479fc55d1d5f9f738b)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:59:36.756856 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3774004f-b460-4874-a1a5-43edcbdcb164 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:59:36.762447 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 994/1991] 10.222.0.33 () {58 vars in 1153 bytes} [Thu Sep 1 14:59:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 14:59:36.763938 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6268876e-0711-4862-8cde-4c1f523cb5d0 admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:59:36.780607 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6268876e-0711-4862-8cde-4c1f523cb5d0 admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:59:36.786556 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f181794-0a19-4b78-ab24-31ff47477756 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:59:36.787182 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f181794-0a19-4b78-ab24-31ff47477756 admin admin] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:36.787462 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f181794-0a19-4b78-ab24-31ff47477756 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:36.787736 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f181794-0a19-4b78-ab24-31ff47477756 admin admin] PATH_INFO: `/v3/projects` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:36.788417 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f181794-0a19-4b78-ab24-31ff47477756 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:59:36.790421 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f181794-0a19-4b78-ab24-31ff47477756 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:59:36.827651 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 995/1992] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Sep 1 14:59:36 2022] POST /identity/v3/projects => generated 375 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 14:59:36.827872 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-3774004f-b460-4874-a1a5-43edcbdcb164 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e86eb68a85b44d479fc55d1d5f9f738b'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:36.829742 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f181794-0a19-4b78-ab24-31ff47477756 admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:59:36.842819 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-3774004f-b460-4874-a1a5-43edcbdcb164 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e86eb68a85b44d479fc55d1d5f9f738b'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:36.843746 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-3774004f-b460-4874-a1a5-43edcbdcb164 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e86eb68a85b44d479fc55d1d5f9f738b'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:36.862909 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-3774004f-b460-4874-a1a5-43edcbdcb164 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e86eb68a85b44d479fc55d1d5f9f738b'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:36.864609 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-3774004f-b460-4874-a1a5-43edcbdcb164 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e86eb68a85b44d479fc55d1d5f9f738b'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:36.883669 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 998/1993] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Sep 1 14:59:36 2022] DELETE /identity/v3/users/e86eb68a85b44d479fc55d1d5f9f738b => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:59:36.885485 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7b222b-32e3-471c-bdc3-8d4b9c9be9e9 None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:36.886519 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7b222b-32e3-471c-bdc3-8d4b9c9be9e9 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:36.886519 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7b222b-32e3-471c-bdc3-8d4b9c9be9e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:36.915475 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-6d7b222b-32e3-471c-bdc3-8d4b9c9be9e9 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:59:36.916870 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-6d7b222b-32e3-471c-bdc3-8d4b9c9be9e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:59:36.936129 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f181794-0a19-4b78-ab24-31ff47477756 admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:59:36.995429 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 999/1994] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 14:59:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 14:59:37.009654 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7b222b-32e3-471c-bdc3-8d4b9c9be9e9 None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:59:37.012605 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-6d7b222b-32e3-471c-bdc3-8d4b9c9be9e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:59:37.029163 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb7d5c9-a102-4f90-9b25-50b631cd91e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:59:37.030207 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb7d5c9-a102-4f90-9b25-50b631cd91e1 admin admin] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:37.030971 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb7d5c9-a102-4f90-9b25-50b631cd91e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:37.031400 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb7d5c9-a102-4f90-9b25-50b631cd91e1 admin admin] PATH_INFO: `/v3/users` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:37.032203 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb7d5c9-a102-4f90-9b25-50b631cd91e1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:59:37.035137 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb7d5c9-a102-4f90-9b25-50b631cd91e1 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:59:37.037247 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:59:37.070309 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 996/1995] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Sep 1 14:59:36 2022] POST /identity/v3/users => generated 288 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:59:37.071405 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb7d5c9-a102-4f90-9b25-50b631cd91e1 admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:59:37.077387 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-6bb7d5c9-a102-4f90-9b25-50b631cd91e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:59:37.107087 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb7d5c9-a102-4f90-9b25-50b631cd91e1 admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:59:37.117317 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d627f625-9f5c-4821-b2a1-33d291039e44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:59:37.119320 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d627f625-9f5c-4821-b2a1-33d291039e44 admin admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:37.119614 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d627f625-9f5c-4821-b2a1-33d291039e44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:37.119910 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d627f625-9f5c-4821-b2a1-33d291039e44 admin admin] PATH_INFO: `/v3/roles` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:37.120539 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d627f625-9f5c-4821-b2a1-33d291039e44 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:59:37.125266 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d627f625-9f5c-4821-b2a1-33d291039e44 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:59:37.136442 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7b222b-32e3-471c-bdc3-8d4b9c9be9e9 None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:59:37.141437 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29704cd8-285e-438b-a2af-fcfb2b75b01a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2790149639aa418bbc4e4982e93eda44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:59:37.141983 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29704cd8-285e-438b-a2af-fcfb2b75b01a service neutron] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:37.142885 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 997/1996] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 1 14:59:37 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:59:37.146164 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29704cd8-285e-438b-a2af-fcfb2b75b01a service neutron] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:37.146505 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29704cd8-285e-438b-a2af-fcfb2b75b01a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:37.147241 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d627f625-9f5c-4821-b2a1-33d291039e44 admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:59:37.158369 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29704cd8-285e-438b-a2af-fcfb2b75b01a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:59:37.163215 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d627f625-9f5c-4821-b2a1-33d291039e44 admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:59:37.163406 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29704cd8-285e-438b-a2af-fcfb2b75b01a service neutron] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:59:37.170500 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90132414-056e-4931-9263-e23b477fea20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:59:37.171429 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90132414-056e-4931-9263-e23b477fea20 admin admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:37.171797 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90132414-056e-4931-9263-e23b477fea20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:37.172170 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90132414-056e-4931-9263-e23b477fea20 admin admin] PATH_INFO: `/v3/roles` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:37.172917 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90132414-056e-4931-9263-e23b477fea20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:59:37.174710 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90132414-056e-4931-9263-e23b477fea20 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:59:37.178167 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1000/1997] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 14:59:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:59:37.192744 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 998/1998] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 1 14:59:37 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 14:59:37.194086 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29704cd8-285e-438b-a2af-fcfb2b75b01a service neutron] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:59:37.208039 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29704cd8-285e-438b-a2af-fcfb2b75b01a service neutron] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:59:37.214475 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c07980-aed5-4b41-a3b5-7ca6a27cd9c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:59:37.215209 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c07980-aed5-4b41-a3b5-7ca6a27cd9c7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:37.216189 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c07980-aed5-4b41-a3b5-7ca6a27cd9c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:37.216573 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c07980-aed5-4b41-a3b5-7ca6a27cd9c7 admin admin] PATH_INFO: `/v3/projects/2684616daed843f09428fdc619aa934f/users/ddf627f6fa19409298cf50aa8d89b4b5/roles/5d1feef0e13d4874928bd86258c4e3b2` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:37.245917 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c07980-aed5-4b41-a3b5-7ca6a27cd9c7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2684616daed843f09428fdc619aa934f, user_id=ddf627f6fa19409298cf50aa8d89b4b5, role_id=5d1feef0e13d4874928bd86258c4e3b2)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:59:37.248858 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c07980-aed5-4b41-a3b5-7ca6a27cd9c7 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:59:37.289171 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1001/1999] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 1 14:59:37 2022] PUT /identity/v3/projects/2684616daed843f09428fdc619aa934f/users/ddf627f6fa19409298cf50aa8d89b4b5/roles/5d1feef0e13d4874928bd86258c4e3b2 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 14:59:37.300965 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b1cbd2-3700-40fb-9821-70832ffb9364 None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:37.301258 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b1cbd2-3700-40fb-9821-70832ffb9364 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:37.301628 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b1cbd2-3700-40fb-9821-70832ffb9364 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:37.335756 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-65b1cbd2-3700-40fb-9821-70832ffb9364 None None] MFA Rules not processed for user `ddf627f6fa19409298cf50aa8d89b4b5`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:59:37.336992 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-65b1cbd2-3700-40fb-9821-70832ffb9364 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:59:37.367405 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:59:37.454293 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 999/2000] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 14:59:37 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:59:37.455572 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c07980-aed5-4b41-a3b5-7ca6a27cd9c7 admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:59:37.575534 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c07980-aed5-4b41-a3b5-7ca6a27cd9c7 admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:59:37.584581 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43125ef-1896-463b-bd2e-c620b3f7acdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:59:37.585357 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43125ef-1896-463b-bd2e-c620b3f7acdf admin admin] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:37.585664 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43125ef-1896-463b-bd2e-c620b3f7acdf admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:37.586014 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43125ef-1896-463b-bd2e-c620b3f7acdf admin admin] PATH_INFO: `/v3/projects` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:37.587027 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43125ef-1896-463b-bd2e-c620b3f7acdf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:59:37.589114 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43125ef-1896-463b-bd2e-c620b3f7acdf admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:59:37.620524 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b1cbd2-3700-40fb-9821-70832ffb9364 None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:59:37.622316 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-65b1cbd2-3700-40fb-9821-70832ffb9364 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:59:37.623476 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1002/2001] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Sep 1 14:59:37 2022] POST /identity/v3/projects => generated 375 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:59:37.631432 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43125ef-1896-463b-bd2e-c620b3f7acdf admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:59:37.644840 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b1cbd2-3700-40fb-9821-70832ffb9364 None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:59:37.714171 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43125ef-1896-463b-bd2e-c620b3f7acdf admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:59:37.718836 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a39732-887a-4f79-9d6f-5f7b4ef899b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:59:37.719465 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a39732-887a-4f79-9d6f-5f7b4ef899b1 admin admin] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:37.719740 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a39732-887a-4f79-9d6f-5f7b4ef899b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:37.720019 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a39732-887a-4f79-9d6f-5f7b4ef899b1 admin admin] PATH_INFO: `/v3/users` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:37.720653 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a39732-887a-4f79-9d6f-5f7b4ef899b1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:59:37.722609 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a39732-887a-4f79-9d6f-5f7b4ef899b1 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:59:37.723155 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb2ad57-d9b7-4c0b-aeee-09426f7cfd96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:59:37.725682 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb2ad57-d9b7-4c0b-aeee-09426f7cfd96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:37.726046 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb2ad57-d9b7-4c0b-aeee-09426f7cfd96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:37.726417 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb2ad57-d9b7-4c0b-aeee-09426f7cfd96 admin admin] PATH_INFO: `/v3/projects/095369e27f2c4bf2bbf1a39dff3957b7` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:37.727653 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb2ad57-d9b7-4c0b-aeee-09426f7cfd96 admin admin] RBAC: Authorizing `identity:delete_project(project_id=095369e27f2c4bf2bbf1a39dff3957b7)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:59:37.729633 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb2ad57-d9b7-4c0b-aeee-09426f7cfd96 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:59:37.749453 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1003/2002] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Sep 1 14:59:37 2022] POST /identity/v3/users => generated 288 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:59:37.754054 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a39732-887a-4f79-9d6f-5f7b4ef899b1 admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:59:37.765774 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a39732-887a-4f79-9d6f-5f7b4ef899b1 admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:59:37.775455 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a0e902-77ee-4b7e-8727-169688575860 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:59:37.776468 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a0e902-77ee-4b7e-8727-169688575860 admin admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:37.776967 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a0e902-77ee-4b7e-8727-169688575860 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:37.777396 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a0e902-77ee-4b7e-8727-169688575860 admin admin] PATH_INFO: `/v3/roles` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:37.778158 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a0e902-77ee-4b7e-8727-169688575860 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:59:37.780069 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a0e902-77ee-4b7e-8727-169688575860 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:59:37.781830 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:59:37.787847 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-abb2ad57-d9b7-4c0b-aeee-09426f7cfd96 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '095369e27f2c4bf2bbf1a39dff3957b7', 'request_id': 'req-abb2ad57-d9b7-4c0b-aeee-09426f7cfd96'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:37.795066 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-abb2ad57-d9b7-4c0b-aeee-09426f7cfd96 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '095369e27f2c4bf2bbf1a39dff3957b7', 'request_id': 'req-abb2ad57-d9b7-4c0b-aeee-09426f7cfd96'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:37.797507 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1004/2003] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 1 14:59:37 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:59:37.801917 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a0e902-77ee-4b7e-8727-169688575860 admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:59:37.805385 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-abb2ad57-d9b7-4c0b-aeee-09426f7cfd96 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '095369e27f2c4bf2bbf1a39dff3957b7', 'request_id': 'req-abb2ad57-d9b7-4c0b-aeee-09426f7cfd96'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:37.812883 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a0e902-77ee-4b7e-8727-169688575860 admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:59:37.818437 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-abb2ad57-d9b7-4c0b-aeee-09426f7cfd96 admin admin] The token cache is being invalidate because project 095369e27f2c4bf2bbf1a39dff3957b7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81341) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:59:37.818917 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-abb2ad57-d9b7-4c0b-aeee-09426f7cfd96 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:37.819553 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-abb2ad57-d9b7-4c0b-aeee-09426f7cfd96 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:37.821277 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1000/2004] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Sep 1 14:59:37 2022] DELETE /identity/v3/projects/095369e27f2c4bf2bbf1a39dff3957b7 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 14:59:37.824925 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb2ad57-d9b7-4c0b-aeee-09426f7cfd96 admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:59:37.826222 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-abb2ad57-d9b7-4c0b-aeee-09426f7cfd96 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:59:37.875902 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd91c275-4589-42f7-b906-8d06bac59d67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:59:37.876525 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd91c275-4589-42f7-b906-8d06bac59d67 admin admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:37.876807 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd91c275-4589-42f7-b906-8d06bac59d67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:37.877084 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd91c275-4589-42f7-b906-8d06bac59d67 admin admin] PATH_INFO: `/v3/roles` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:37.878601 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd91c275-4589-42f7-b906-8d06bac59d67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:59:37.880243 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd91c275-4589-42f7-b906-8d06bac59d67 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:59:37.894590 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1005/2005] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 1 14:59:37 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 14:59:37.896246 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd91c275-4589-42f7-b906-8d06bac59d67 admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:59:37.896634 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:59:37.898362 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-cd91c275-4589-42f7-b906-8d06bac59d67 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:59:37.901775 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb2ad57-d9b7-4c0b-aeee-09426f7cfd96 admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:59:37.910531 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716ef9ee-020a-40d4-b1a0-aa7d1b1cfd70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:59:37.911291 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716ef9ee-020a-40d4-b1a0-aa7d1b1cfd70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:37.911827 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716ef9ee-020a-40d4-b1a0-aa7d1b1cfd70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:37.912171 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716ef9ee-020a-40d4-b1a0-aa7d1b1cfd70 admin admin] PATH_INFO: `/v3/users/66c59917198a4a8fbfc20dc9b9b95514` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:37.912392 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd91c275-4589-42f7-b906-8d06bac59d67 admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:59:37.913562 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716ef9ee-020a-40d4-b1a0-aa7d1b1cfd70 admin admin] RBAC: Authorizing `identity:delete_user(user_id=66c59917198a4a8fbfc20dc9b9b95514)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:59:37.915725 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716ef9ee-020a-40d4-b1a0-aa7d1b1cfd70 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:59:37.917605 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c174fa0-51f1-4d66-a4ea-bd448e467279 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:59:37.918568 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c174fa0-51f1-4d66-a4ea-bd448e467279 admin admin] REQUEST_METHOD: `PUT` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:37.918970 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c174fa0-51f1-4d66-a4ea-bd448e467279 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:37.919245 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c174fa0-51f1-4d66-a4ea-bd448e467279 admin admin] PATH_INFO: `/v3/projects/a5897696fdbf4cd7a72330f7327c85dc/users/34c146d16faa4ddcbb1956535303214a/roles/5d1feef0e13d4874928bd86258c4e3b2` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:37.943736 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c174fa0-51f1-4d66-a4ea-bd448e467279 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a5897696fdbf4cd7a72330f7327c85dc, user_id=34c146d16faa4ddcbb1956535303214a, role_id=5d1feef0e13d4874928bd86258c4e3b2)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:59:37.947364 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c174fa0-51f1-4d66-a4ea-bd448e467279 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:59:37.963754 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1006/2006] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 1 14:59:37 2022] PUT /identity/v3/projects/a5897696fdbf4cd7a72330f7327c85dc/users/34c146d16faa4ddcbb1956535303214a/roles/5d1feef0e13d4874928bd86258c4e3b2 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 14:59:37.976577 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f990875f-0e18-4b79-bafe-2c8008237f8a None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:37.976960 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f990875f-0e18-4b79-bafe-2c8008237f8a None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:37.977302 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f990875f-0e18-4b79-bafe-2c8008237f8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:37.992069 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-716ef9ee-020a-40d4-b1a0-aa7d1b1cfd70 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '66c59917198a4a8fbfc20dc9b9b95514'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:38.003840 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-716ef9ee-020a-40d4-b1a0-aa7d1b1cfd70 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '66c59917198a4a8fbfc20dc9b9b95514'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:38.004690 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-716ef9ee-020a-40d4-b1a0-aa7d1b1cfd70 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '66c59917198a4a8fbfc20dc9b9b95514'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:38.021913 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-f990875f-0e18-4b79-bafe-2c8008237f8a None None] MFA Rules not processed for user `34c146d16faa4ddcbb1956535303214a`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:59:38.025003 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-716ef9ee-020a-40d4-b1a0-aa7d1b1cfd70 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '66c59917198a4a8fbfc20dc9b9b95514'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:38.026182 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-716ef9ee-020a-40d4-b1a0-aa7d1b1cfd70 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '66c59917198a4a8fbfc20dc9b9b95514'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:38.030334 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-f990875f-0e18-4b79-bafe-2c8008237f8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:59:38.045038 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1001/2007] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Sep 1 14:59:37 2022] DELETE /identity/v3/users/66c59917198a4a8fbfc20dc9b9b95514 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 14:59:38.130000 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1007/2008] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 14:59:37 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 14:59:38.134297 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716ef9ee-020a-40d4-b1a0-aa7d1b1cfd70 admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:59:38.135933 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-716ef9ee-020a-40d4-b1a0-aa7d1b1cfd70 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:59:38.229940 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716ef9ee-020a-40d4-b1a0-aa7d1b1cfd70 admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:59:38.235393 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682daae9-ce9e-447e-bd00-d76f6f08d2c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:59:38.236162 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682daae9-ce9e-447e-bd00-d76f6f08d2c0 admin admin] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:38.236589 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682daae9-ce9e-447e-bd00-d76f6f08d2c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:38.237621 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682daae9-ce9e-447e-bd00-d76f6f08d2c0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:38.238341 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682daae9-ce9e-447e-bd00-d76f6f08d2c0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:59:38.240366 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682daae9-ce9e-447e-bd00-d76f6f08d2c0 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:59:38.282151 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1002/2009] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Sep 1 14:59:38 2022] POST /identity/v3/projects => generated 373 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:59:38.283456 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f990875f-0e18-4b79-bafe-2c8008237f8a None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:59:38.373180 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:59:38.387956 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f990875f-0e18-4b79-bafe-2c8008237f8a None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:59:38.390050 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682daae9-ce9e-447e-bd00-d76f6f08d2c0 admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:59:38.392054 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-682daae9-ce9e-447e-bd00-d76f6f08d2c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:59:38.483041 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8c12ad-377b-4523-8dc2-1293db8a998c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:59:38.484052 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8c12ad-377b-4523-8dc2-1293db8a998c admin admin] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:38.484052 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8c12ad-377b-4523-8dc2-1293db8a998c admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:38.484421 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8c12ad-377b-4523-8dc2-1293db8a998c admin admin] PATH_INFO: `/v3/users` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:38.485156 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8c12ad-377b-4523-8dc2-1293db8a998c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:59:38.487437 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8c12ad-377b-4523-8dc2-1293db8a998c admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:59:38.492616 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682daae9-ce9e-447e-bd00-d76f6f08d2c0 admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:59:38.497713 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f70893-26db-4e26-90ee-aec481dbcc93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:59:38.498430 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f70893-26db-4e26-90ee-aec481dbcc93 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:38.498813 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f70893-26db-4e26-90ee-aec481dbcc93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:38.499093 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f70893-26db-4e26-90ee-aec481dbcc93 admin admin] PATH_INFO: `/v3/projects/d7acaee8811a4cdba43a7fb3e9610b26` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:38.500316 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f70893-26db-4e26-90ee-aec481dbcc93 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d7acaee8811a4cdba43a7fb3e9610b26)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:59:38.502477 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f70893-26db-4e26-90ee-aec481dbcc93 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:59:38.505215 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:59:38.515439 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1008/2010] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Sep 1 14:59:38 2022] POST /identity/v3/users => generated 293 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:59:38.519984 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8c12ad-377b-4523-8dc2-1293db8a998c admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:59:38.530841 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8c12ad-377b-4523-8dc2-1293db8a998c admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:59:38.536651 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd6ca9b-cc46-4963-a762-4b99d7fd29b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:59:38.537690 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd6ca9b-cc46-4963-a762-4b99d7fd29b2 admin admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:38.538392 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd6ca9b-cc46-4963-a762-4b99d7fd29b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:38.538591 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd6ca9b-cc46-4963-a762-4b99d7fd29b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:38.539345 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd6ca9b-cc46-4963-a762-4b99d7fd29b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:59:38.541204 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd6ca9b-cc46-4963-a762-4b99d7fd29b2 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:59:38.556189 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1009/2011] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 1 14:59:38 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 14:59:38.557373 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd6ca9b-cc46-4963-a762-4b99d7fd29b2 admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:59:38.567650 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd6ca9b-cc46-4963-a762-4b99d7fd29b2 admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:59:38.569428 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-94f70893-26db-4e26-90ee-aec481dbcc93 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd7acaee8811a4cdba43a7fb3e9610b26', 'request_id': 'req-94f70893-26db-4e26-90ee-aec481dbcc93'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:38.577246 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-94f70893-26db-4e26-90ee-aec481dbcc93 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd7acaee8811a4cdba43a7fb3e9610b26', 'request_id': 'req-94f70893-26db-4e26-90ee-aec481dbcc93'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:38.585987 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-94f70893-26db-4e26-90ee-aec481dbcc93 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd7acaee8811a4cdba43a7fb3e9610b26', 'request_id': 'req-94f70893-26db-4e26-90ee-aec481dbcc93'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:38.598460 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-94f70893-26db-4e26-90ee-aec481dbcc93 admin admin] The token cache is being invalidate because project d7acaee8811a4cdba43a7fb3e9610b26 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81341) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:59:38.598858 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-94f70893-26db-4e26-90ee-aec481dbcc93 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:38.599583 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-94f70893-26db-4e26-90ee-aec481dbcc93 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:38.601174 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1003/2012] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Sep 1 14:59:38 2022] DELETE /identity/v3/projects/d7acaee8811a4cdba43a7fb3e9610b26 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:59:38.606756 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698c38be-d45c-494f-a9f7-585412b23cb8 None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:38.607219 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698c38be-d45c-494f-a9f7-585412b23cb8 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:38.607615 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698c38be-d45c-494f-a9f7-585412b23cb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:38.629897 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-698c38be-d45c-494f-a9f7-585412b23cb8 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:59:38.631406 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-698c38be-d45c-494f-a9f7-585412b23cb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:59:38.642473 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7d9701-66b6-44ee-b401-8f6b3cfc28d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:59:38.642923 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7d9701-66b6-44ee-b401-8f6b3cfc28d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:38.643736 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7d9701-66b6-44ee-b401-8f6b3cfc28d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:38.644008 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7d9701-66b6-44ee-b401-8f6b3cfc28d3 admin admin] PATH_INFO: `/v3/projects/1fa7856e698843d1917437f0a2cbbeb2/users/62d6b8950b26429d83eff77c2edaf1d1/roles/2a2adb769e55476b82e5014897e0a9c0` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:38.670262 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7d9701-66b6-44ee-b401-8f6b3cfc28d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1fa7856e698843d1917437f0a2cbbeb2, user_id=62d6b8950b26429d83eff77c2edaf1d1, role_id=2a2adb769e55476b82e5014897e0a9c0)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:59:38.673829 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7d9701-66b6-44ee-b401-8f6b3cfc28d3 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:59:38.689574 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1010/2013] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 1 14:59:38 2022] PUT /identity/v3/projects/1fa7856e698843d1917437f0a2cbbeb2/users/62d6b8950b26429d83eff77c2edaf1d1/roles/2a2adb769e55476b82e5014897e0a9c0 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:59:38.694259 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7d9701-66b6-44ee-b401-8f6b3cfc28d3 admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:59:38.695939 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-cb7d9701-66b6-44ee-b401-8f6b3cfc28d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:59:38.704765 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1004/2014] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 14:59:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 14:59:38.714556 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1487e508-bf5d-41df-9060-841d3449fef4 None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:38.714986 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1487e508-bf5d-41df-9060-841d3449fef4 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:38.715356 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1487e508-bf5d-41df-9060-841d3449fef4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:38.736848 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-1487e508-bf5d-41df-9060-841d3449fef4 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:59:38.737979 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-1487e508-bf5d-41df-9060-841d3449fef4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:59:38.764696 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7d9701-66b6-44ee-b401-8f6b3cfc28d3 admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:59:38.770419 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcba740-96a2-4b2a-9d72-8aa60983eae8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:59:38.771251 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcba740-96a2-4b2a-9d72-8aa60983eae8 admin admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:38.771546 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcba740-96a2-4b2a-9d72-8aa60983eae8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:38.771818 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcba740-96a2-4b2a-9d72-8aa60983eae8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:38.772735 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcba740-96a2-4b2a-9d72-8aa60983eae8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:59:38.774732 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcba740-96a2-4b2a-9d72-8aa60983eae8 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:59:38.786198 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1011/2015] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 1 14:59:38 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 14:59:38.790429 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcba740-96a2-4b2a-9d72-8aa60983eae8 admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:59:38.795633 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:59:38.799344 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1005/2016] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 14:59:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:59:38.803589 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1487e508-bf5d-41df-9060-841d3449fef4 None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:59:38.804728 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcba740-96a2-4b2a-9d72-8aa60983eae8 admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:59:38.811044 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b617b4-f3f9-4df5-aabf-d39ea80c5708 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:59:38.812111 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b617b4-f3f9-4df5-aabf-d39ea80c5708 admin admin] REQUEST_METHOD: `PUT` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:38.812111 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b617b4-f3f9-4df5-aabf-d39ea80c5708 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:38.812330 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b617b4-f3f9-4df5-aabf-d39ea80c5708 admin admin] PATH_INFO: `/v3/projects/1fa7856e698843d1917437f0a2cbbeb2/users/62d6b8950b26429d83eff77c2edaf1d1/roles/2a2adb769e55476b82e5014897e0a9c0` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:38.812980 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1487e508-bf5d-41df-9060-841d3449fef4 None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:59:38.814157 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b617b4-f3f9-4df5-aabf-d39ea80c5708 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1fa7856e698843d1917437f0a2cbbeb2, user_id=62d6b8950b26429d83eff77c2edaf1d1, role_id=2a2adb769e55476b82e5014897e0a9c0)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:59:38.817556 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b617b4-f3f9-4df5-aabf-d39ea80c5708 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:59:38.818615 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cc3209-9bab-4129-9663-83c5010e3a1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:59:38.819243 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cc3209-9bab-4129-9663-83c5010e3a1f admin admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:38.819520 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cc3209-9bab-4129-9663-83c5010e3a1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:38.819830 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cc3209-9bab-4129-9663-83c5010e3a1f admin admin] PATH_INFO: `/v3/domains` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:38.820552 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71cc3209-9bab-4129-9663-83c5010e3a1f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81341) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:59:38.820929 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71cc3209-9bab-4129-9663-83c5010e3a1f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:59:38.822690 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71cc3209-9bab-4129-9663-83c5010e3a1f admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:59:38.828659 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1012/2017] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 1 14:59:38 2022] PUT /identity/v3/projects/1fa7856e698843d1917437f0a2cbbeb2/users/62d6b8950b26429d83eff77c2edaf1d1/roles/2a2adb769e55476b82e5014897e0a9c0 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 14:59:38.835825 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1006/2018] 10.222.0.33 () {58 vars in 1153 bytes} [Thu Sep 1 14:59:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 14:59:38.839158 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbab3f7-ccf3-4870-a2fb-961475db5bb7 None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:38.839522 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbab3f7-ccf3-4870-a2fb-961475db5bb7 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:38.839794 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbab3f7-ccf3-4870-a2fb-961475db5bb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:38.839952 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cc3209-9bab-4129-9663-83c5010e3a1f admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:59:38.850922 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cc3209-9bab-4129-9663-83c5010e3a1f admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:59:38.869859 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-1cbab3f7-ccf3-4870-a2fb-961475db5bb7 None None] MFA Rules not processed for user `62d6b8950b26429d83eff77c2edaf1d1`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:59:38.871122 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-1cbab3f7-ccf3-4870-a2fb-961475db5bb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:59:38.872922 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:59:38.915276 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a808721b-52db-4dcc-99f6-ff1fc7b11215 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:59:38.915888 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a808721b-52db-4dcc-99f6-ff1fc7b11215 admin admin] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:38.916173 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a808721b-52db-4dcc-99f6-ff1fc7b11215 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:38.916448 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a808721b-52db-4dcc-99f6-ff1fc7b11215 admin admin] PATH_INFO: `/v3/projects` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:38.917074 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a808721b-52db-4dcc-99f6-ff1fc7b11215 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:59:38.919126 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a808721b-52db-4dcc-99f6-ff1fc7b11215 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:59:38.943859 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1013/2019] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 14:59:38 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 14:59:38.953820 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1007/2020] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Sep 1 14:59:38 2022] POST /identity/v3/projects => generated 409 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 14:59:38.957908 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbab3f7-ccf3-4870-a2fb-961475db5bb7 None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:59:38.958214 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a808721b-52db-4dcc-99f6-ff1fc7b11215 admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:59:38.959865 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-a808721b-52db-4dcc-99f6-ff1fc7b11215 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:59:38.968100 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbab3f7-ccf3-4870-a2fb-961475db5bb7 None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:59:39.015784 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a808721b-52db-4dcc-99f6-ff1fc7b11215 admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:59:39.022570 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c78a7e2-02c7-43e2-8143-298e41da1f90 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3b81739e34e4e648f88b6d73fcb5e46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:59:39.023196 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c78a7e2-02c7-43e2-8143-298e41da1f90 service cinder] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:39.023473 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c78a7e2-02c7-43e2-8143-298e41da1f90 service cinder] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:39.023745 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c78a7e2-02c7-43e2-8143-298e41da1f90 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:39.025513 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-2c78a7e2-02c7-43e2-8143-298e41da1f90 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:59:39.028057 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e89aef8-b633-44ea-9287-5683db81657b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:59:39.028667 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e89aef8-b633-44ea-9287-5683db81657b admin admin] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:39.028948 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e89aef8-b633-44ea-9287-5683db81657b admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:39.029219 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e89aef8-b633-44ea-9287-5683db81657b admin admin] PATH_INFO: `/v3/users` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:39.029867 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e89aef8-b633-44ea-9287-5683db81657b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:59:39.031837 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e89aef8-b633-44ea-9287-5683db81657b admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:59:39.055261 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1014/2021] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Sep 1 14:59:38 2022] POST /identity/v3/users => generated 305 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:59:39.059773 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e89aef8-b633-44ea-9287-5683db81657b admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:59:39.069793 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e89aef8-b633-44ea-9287-5683db81657b admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:59:39.074492 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3d5c29-9207-4bcc-ab1b-4c255eb4395b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:59:39.075079 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3d5c29-9207-4bcc-ab1b-4c255eb4395b admin admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:39.075358 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3d5c29-9207-4bcc-ab1b-4c255eb4395b admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:39.075630 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3d5c29-9207-4bcc-ab1b-4c255eb4395b admin admin] PATH_INFO: `/v3/roles` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:39.076387 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3d5c29-9207-4bcc-ab1b-4c255eb4395b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:59:39.077979 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3d5c29-9207-4bcc-ab1b-4c255eb4395b admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:59:39.086853 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c78a7e2-02c7-43e2-8143-298e41da1f90 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:59:39.088270 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c78a7e2-02c7-43e2-8143-298e41da1f90 service cinder] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:59:39.090005 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1015/2022] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 1 14:59:39 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:59:39.093433 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3d5c29-9207-4bcc-ab1b-4c255eb4395b admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:59:39.104676 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1008/2023] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 14:59:38 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 14:59:39.104945 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3d5c29-9207-4bcc-ab1b-4c255eb4395b admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:59:39.110381 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4580cf09-21e6-46c2-87c2-e99b4c1c77d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:59:39.110986 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4580cf09-21e6-46c2-87c2-e99b4c1c77d4 admin admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:39.111255 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4580cf09-21e6-46c2-87c2-e99b4c1c77d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:39.111527 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4580cf09-21e6-46c2-87c2-e99b4c1c77d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:39.112230 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4580cf09-21e6-46c2-87c2-e99b4c1c77d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:59:39.113941 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4580cf09-21e6-46c2-87c2-e99b4c1c77d4 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:59:39.126603 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1016/2024] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 1 14:59:39 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 14:59:39.130911 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c78a7e2-02c7-43e2-8143-298e41da1f90 service cinder] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:59:39.144444 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c78a7e2-02c7-43e2-8143-298e41da1f90 service cinder] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:59:39.151905 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f323e4-681e-4c89-9be4-8c9f4a4afd58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:59:39.152689 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f323e4-681e-4c89-9be4-8c9f4a4afd58 admin admin] REQUEST_METHOD: `PUT` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:39.153135 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f323e4-681e-4c89-9be4-8c9f4a4afd58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:39.153468 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f323e4-681e-4c89-9be4-8c9f4a4afd58 admin admin] PATH_INFO: `/v3/projects/baf1a0cc82104a8ea760e2e861bf4036/users/619b17c6d75749eba9fbfda01d26c292/roles/5d1feef0e13d4874928bd86258c4e3b2` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:39.170388 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f323e4-681e-4c89-9be4-8c9f4a4afd58 admin admin] RBAC: Authorizing `identity:create_grant(project_id=baf1a0cc82104a8ea760e2e861bf4036, user_id=619b17c6d75749eba9fbfda01d26c292, role_id=5d1feef0e13d4874928bd86258c4e3b2)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:59:39.173024 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f323e4-681e-4c89-9be4-8c9f4a4afd58 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:59:39.189028 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1009/2025] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 1 14:59:39 2022] PUT /identity/v3/projects/baf1a0cc82104a8ea760e2e861bf4036/users/619b17c6d75749eba9fbfda01d26c292/roles/5d1feef0e13d4874928bd86258c4e3b2 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:59:39.202830 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf33852-017b-4f90-ae1f-44b93e38f42f None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:39.202830 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf33852-017b-4f90-ae1f-44b93e38f42f None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:39.202830 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf33852-017b-4f90-ae1f-44b93e38f42f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:39.233894 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-3cf33852-017b-4f90-ae1f-44b93e38f42f None None] MFA Rules not processed for user `619b17c6d75749eba9fbfda01d26c292`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:59:39.235236 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-3cf33852-017b-4f90-ae1f-44b93e38f42f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:59:39.275771 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:59:39.320101 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1017/2026] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 14:59:39 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:59:39.324608 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f323e4-681e-4c89-9be4-8c9f4a4afd58 admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:59:39.377362 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:59:39.414501 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f323e4-681e-4c89-9be4-8c9f4a4afd58 admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:59:39.416788 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798784d8-d231-4463-b4c2-ebc825c96b1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:59:39.417389 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798784d8-d231-4463-b4c2-ebc825c96b1e admin admin] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:39.417668 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798784d8-d231-4463-b4c2-ebc825c96b1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:39.417936 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798784d8-d231-4463-b4c2-ebc825c96b1e admin admin] PATH_INFO: `/v3/projects` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:39.418759 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798784d8-d231-4463-b4c2-ebc825c96b1e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:59:39.426132 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798784d8-d231-4463-b4c2-ebc825c96b1e admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:59:39.460432 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1010/2027] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Sep 1 14:59:39 2022] POST /identity/v3/projects => generated 409 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 14:59:39.464459 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf33852-017b-4f90-ae1f-44b93e38f42f None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:59:39.477031 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf33852-017b-4f90-ae1f-44b93e38f42f None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:59:39.566783 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729f0940-694b-40d7-a062-f582b25a624c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:59:39.567392 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729f0940-694b-40d7-a062-f582b25a624c admin admin] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:39.567720 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729f0940-694b-40d7-a062-f582b25a624c admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:39.567923 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729f0940-694b-40d7-a062-f582b25a624c admin admin] PATH_INFO: `/v3/users` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:39.568688 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729f0940-694b-40d7-a062-f582b25a624c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:59:39.571525 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729f0940-694b-40d7-a062-f582b25a624c admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:59:39.595282 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1018/2028] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Sep 1 14:59:39 2022] POST /identity/v3/users => generated 311 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 14:59:39.600216 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798784d8-d231-4463-b4c2-ebc825c96b1e admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:59:39.610490 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798784d8-d231-4463-b4c2-ebc825c96b1e admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:59:39.615672 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f27efe-a68a-4a33-9fab-4ff42fdf9df1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:59:39.616265 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f27efe-a68a-4a33-9fab-4ff42fdf9df1 admin admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:39.616541 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f27efe-a68a-4a33-9fab-4ff42fdf9df1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:39.616809 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f27efe-a68a-4a33-9fab-4ff42fdf9df1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:39.617700 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f27efe-a68a-4a33-9fab-4ff42fdf9df1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:59:39.619120 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f27efe-a68a-4a33-9fab-4ff42fdf9df1 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:59:39.631163 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1011/2029] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 1 14:59:39 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:59:39.636015 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729f0940-694b-40d7-a062-f582b25a624c admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:59:39.646393 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729f0940-694b-40d7-a062-f582b25a624c admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:59:39.653606 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2c6cd1-45ad-457a-9a5a-af49ce05b2bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:59:39.654304 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2c6cd1-45ad-457a-9a5a-af49ce05b2bc admin admin] REQUEST_METHOD: `PUT` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:39.654717 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2c6cd1-45ad-457a-9a5a-af49ce05b2bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:39.655038 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2c6cd1-45ad-457a-9a5a-af49ce05b2bc admin admin] PATH_INFO: `/v3/projects/d30ddfb1adee468bba325b4d823778fe/users/2e23d8d81e9841ada564a1481680a97e/roles/2a2adb769e55476b82e5014897e0a9c0` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:39.676307 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2c6cd1-45ad-457a-9a5a-af49ce05b2bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=d30ddfb1adee468bba325b4d823778fe, user_id=2e23d8d81e9841ada564a1481680a97e, role_id=2a2adb769e55476b82e5014897e0a9c0)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:59:39.679335 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2c6cd1-45ad-457a-9a5a-af49ce05b2bc admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:59:39.697851 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1019/2030] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 1 14:59:39 2022] PUT /identity/v3/projects/d30ddfb1adee468bba325b4d823778fe/users/2e23d8d81e9841ada564a1481680a97e/roles/2a2adb769e55476b82e5014897e0a9c0 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 14:59:39.699588 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f27efe-a68a-4a33-9fab-4ff42fdf9df1 admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:59:39.709993 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f27efe-a68a-4a33-9fab-4ff42fdf9df1 admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:59:39.766833 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e917e27a-0ac9-49f3-bab4-dc9b1815a502 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:59:39.767440 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e917e27a-0ac9-49f3-bab4-dc9b1815a502 admin admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:39.767726 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e917e27a-0ac9-49f3-bab4-dc9b1815a502 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:39.768028 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e917e27a-0ac9-49f3-bab4-dc9b1815a502 admin admin] PATH_INFO: `/v3/roles` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:39.768664 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e917e27a-0ac9-49f3-bab4-dc9b1815a502 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:59:39.770294 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e917e27a-0ac9-49f3-bab4-dc9b1815a502 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:59:39.779901 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1012/2031] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 1 14:59:39 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:59:39.784660 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2c6cd1-45ad-457a-9a5a-af49ce05b2bc admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:59:39.794868 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2c6cd1-45ad-457a-9a5a-af49ce05b2bc admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:59:39.800204 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a34283-88b7-4345-b61b-fd20e642e5c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:59:39.800846 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a34283-88b7-4345-b61b-fd20e642e5c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:39.801124 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a34283-88b7-4345-b61b-fd20e642e5c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:39.801395 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a34283-88b7-4345-b61b-fd20e642e5c6 admin admin] PATH_INFO: `/v3/projects/d30ddfb1adee468bba325b4d823778fe/users/2e23d8d81e9841ada564a1481680a97e/roles/2a2adb769e55476b82e5014897e0a9c0` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:39.803598 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a34283-88b7-4345-b61b-fd20e642e5c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d30ddfb1adee468bba325b4d823778fe, user_id=2e23d8d81e9841ada564a1481680a97e, role_id=2a2adb769e55476b82e5014897e0a9c0)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:59:39.806716 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a34283-88b7-4345-b61b-fd20e642e5c6 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:59:39.820036 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1020/2032] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 1 14:59:39 2022] PUT /identity/v3/projects/d30ddfb1adee468bba325b4d823778fe/users/2e23d8d81e9841ada564a1481680a97e/roles/2a2adb769e55476b82e5014897e0a9c0 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:59:39.832589 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f619f42e-51ed-4ab9-b1ff-60448af24dc8 None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:39.832868 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f619f42e-51ed-4ab9-b1ff-60448af24dc8 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:39.833141 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f619f42e-51ed-4ab9-b1ff-60448af24dc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:39.861947 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-f619f42e-51ed-4ab9-b1ff-60448af24dc8 None None] MFA Rules not processed for user `2e23d8d81e9841ada564a1481680a97e`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:59:39.863201 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-f619f42e-51ed-4ab9-b1ff-60448af24dc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:59:39.933349 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1013/2033] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 14:59:39 2022] POST /identity/v3/auth/tokens => generated 3353 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:59:39.944519 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a34283-88b7-4345-b61b-fd20e642e5c6 admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:59:39.946205 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-e9a34283-88b7-4345-b61b-fd20e642e5c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:59:40.006728 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a34283-88b7-4345-b61b-fd20e642e5c6 admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:59:40.011011 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e594f0d-20bf-48f8-b4a9-191fa2a9534c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3b81739e34e4e648f88b6d73fcb5e46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:59:40.011672 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e594f0d-20bf-48f8-b4a9-191fa2a9534c service cinder] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:40.011952 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e594f0d-20bf-48f8-b4a9-191fa2a9534c service cinder] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:40.012225 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e594f0d-20bf-48f8-b4a9-191fa2a9534c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:40.021817 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e594f0d-20bf-48f8-b4a9-191fa2a9534c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:59:40.023299 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e594f0d-20bf-48f8-b4a9-191fa2a9534c service cinder] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:59:40.034702 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1021/2034] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 14:59:39 2022] GET /identity/v3/auth/tokens => generated 3353 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:59:40.194818 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f619f42e-51ed-4ab9-b1ff-60448af24dc8 None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:59:40.204476 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f619f42e-51ed-4ab9-b1ff-60448af24dc8 None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:59:40.208694 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e2e3d1-ad59-4f8f-8913-4953bc213dee service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3b81739e34e4e648f88b6d73fcb5e46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:59:40.209281 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e2e3d1-ad59-4f8f-8913-4953bc213dee service cinder] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:40.209561 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e2e3d1-ad59-4f8f-8913-4953bc213dee service cinder] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:40.209860 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e2e3d1-ad59-4f8f-8913-4953bc213dee service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:40.238177 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:59:40.266876 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e2e3d1-ad59-4f8f-8913-4953bc213dee service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:59:40.268909 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e2e3d1-ad59-4f8f-8913-4953bc213dee service cinder] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:59:40.277976 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1014/2035] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 14:59:40 2022] GET /identity/v3/auth/tokens => generated 3286 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:59:40.742586 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e594f0d-20bf-48f8-b4a9-191fa2a9534c service cinder] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:59:40.751108 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e594f0d-20bf-48f8-b4a9-191fa2a9534c service cinder] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:59:40.756573 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fecb25-3276-4151-b363-7927cebaa439 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3b81739e34e4e648f88b6d73fcb5e46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:59:40.757174 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fecb25-3276-4151-b363-7927cebaa439 service cinder] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:40.757459 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fecb25-3276-4151-b363-7927cebaa439 service cinder] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:40.757713 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fecb25-3276-4151-b363-7927cebaa439 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:40.759444 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-91fecb25-3276-4151-b363-7927cebaa439 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:59:40.797128 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:59:40.843274 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fecb25-3276-4151-b363-7927cebaa439 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:59:40.844735 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fecb25-3276-4151-b363-7927cebaa439 service cinder] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:59:40.859669 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1022/2036] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 14:59:40 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 14:59:41.475853 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e2e3d1-ad59-4f8f-8913-4953bc213dee service cinder] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:59:41.484658 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e2e3d1-ad59-4f8f-8913-4953bc213dee service cinder] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:59:41.488913 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57713a2d-c279-4761-b7d7-7522087e741a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3b81739e34e4e648f88b6d73fcb5e46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:59:41.489526 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57713a2d-c279-4761-b7d7-7522087e741a service cinder] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:41.489802 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57713a2d-c279-4761-b7d7-7522087e741a service cinder] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:41.490069 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57713a2d-c279-4761-b7d7-7522087e741a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:41.556177 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57713a2d-c279-4761-b7d7-7522087e741a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:59:41.557686 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57713a2d-c279-4761-b7d7-7522087e741a service cinder] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:59:41.569398 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1015/2037] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 14:59:41 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:59:44.263183 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943d7df1-68b9-4371-b575-8fc508e4b0f3 None None] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:44.263765 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943d7df1-68b9-4371-b575-8fc508e4b0f3 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:44.263867 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943d7df1-68b9-4371-b575-8fc508e4b0f3 None None] PATH_INFO: `/` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:44.265581 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1023/2038] 10.222.0.33 () {54 vars in 921 bytes} [Thu Sep 1 14:59:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:59:44.276611 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f904c33-07a8-4d54-bbb8-3632005a8f99 None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:44.277114 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f904c33-07a8-4d54-bbb8-3632005a8f99 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:44.277213 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f904c33-07a8-4d54-bbb8-3632005a8f99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:44.300065 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-4f904c33-07a8-4d54-bbb8-3632005a8f99 None None] MFA Rules not processed for user `7f312c8dcabe48ef9ea453364ade38ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:59:44.301428 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-4f904c33-07a8-4d54-bbb8-3632005a8f99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:59:44.338745 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943d7df1-68b9-4371-b575-8fc508e4b0f3 None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:59:44.340498 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-943d7df1-68b9-4371-b575-8fc508e4b0f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:59:44.370700 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1016/2039] 10.222.0.33 () {58 vars in 1018 bytes} [Thu Sep 1 14:59:44 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 14:59:44.382116 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f904c33-07a8-4d54-bbb8-3632005a8f99 None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:59:44.388879 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-4f904c33-07a8-4d54-bbb8-3632005a8f99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:59:44.396078 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f904c33-07a8-4d54-bbb8-3632005a8f99 None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:59:44.402178 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d1683f-2bb4-4ff2-9ab4-236702294837 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f312c8dcabe48ef9ea453364ade38ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:59:44.402850 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d1683f-2bb4-4ff2-9ab4-236702294837 service placement] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:44.403130 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d1683f-2bb4-4ff2-9ab4-236702294837 service placement] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:44.403402 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d1683f-2bb4-4ff2-9ab4-236702294837 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:44.413728 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d1683f-2bb4-4ff2-9ab4-236702294837 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:59:44.415434 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d1683f-2bb4-4ff2-9ab4-236702294837 service placement] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:59:44.416432 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943d7df1-68b9-4371-b575-8fc508e4b0f3 None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:59:44.422472 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b29869-26d3-4ba4-bb50-de7e76c2ddfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:59:44.423223 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b29869-26d3-4ba4-bb50-de7e76c2ddfc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:44.423583 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b29869-26d3-4ba4-bb50-de7e76c2ddfc admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:44.424558 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b29869-26d3-4ba4-bb50-de7e76c2ddfc admin admin] PATH_INFO: `/v3/users/bb7098b844d743f5bafeb7ba15b3fdd7` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:44.425846 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b29869-26d3-4ba4-bb50-de7e76c2ddfc admin admin] RBAC: Authorizing `identity:delete_user(user_id=bb7098b844d743f5bafeb7ba15b3fdd7)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:59:44.428137 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b29869-26d3-4ba4-bb50-de7e76c2ddfc admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:59:44.428710 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1017/2040] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 14:59:44 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 14:59:44.481897 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-72b29869-26d3-4ba4-bb50-de7e76c2ddfc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bb7098b844d743f5bafeb7ba15b3fdd7'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:44.491885 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-72b29869-26d3-4ba4-bb50-de7e76c2ddfc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bb7098b844d743f5bafeb7ba15b3fdd7'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:44.493132 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-72b29869-26d3-4ba4-bb50-de7e76c2ddfc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bb7098b844d743f5bafeb7ba15b3fdd7'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:44.507892 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-72b29869-26d3-4ba4-bb50-de7e76c2ddfc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bb7098b844d743f5bafeb7ba15b3fdd7'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:44.508633 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-72b29869-26d3-4ba4-bb50-de7e76c2ddfc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bb7098b844d743f5bafeb7ba15b3fdd7'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:44.522806 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1024/2041] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Sep 1 14:59:44 2022] DELETE /identity/v3/users/bb7098b844d743f5bafeb7ba15b3fdd7 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 14:59:44.695023 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d1683f-2bb4-4ff2-9ab4-236702294837 service placement] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:59:44.706267 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d1683f-2bb4-4ff2-9ab4-236702294837 service placement] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:59:44.758770 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:59:44.771344 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20a5788-7ba6-49ae-8af2-9edbe3407f28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:59:44.771972 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20a5788-7ba6-49ae-8af2-9edbe3407f28 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:44.772252 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20a5788-7ba6-49ae-8af2-9edbe3407f28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:44.772524 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20a5788-7ba6-49ae-8af2-9edbe3407f28 admin admin] PATH_INFO: `/v3/projects/d925def3f55b435bb5452a0b5e5daff5` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:44.773672 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20a5788-7ba6-49ae-8af2-9edbe3407f28 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d925def3f55b435bb5452a0b5e5daff5)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:59:44.775630 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20a5788-7ba6-49ae-8af2-9edbe3407f28 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:59:44.822545 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-f20a5788-7ba6-49ae-8af2-9edbe3407f28 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd925def3f55b435bb5452a0b5e5daff5', 'request_id': 'req-f20a5788-7ba6-49ae-8af2-9edbe3407f28'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:44.828790 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-f20a5788-7ba6-49ae-8af2-9edbe3407f28 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd925def3f55b435bb5452a0b5e5daff5', 'request_id': 'req-f20a5788-7ba6-49ae-8af2-9edbe3407f28'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:44.836614 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-f20a5788-7ba6-49ae-8af2-9edbe3407f28 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd925def3f55b435bb5452a0b5e5daff5', 'request_id': 'req-f20a5788-7ba6-49ae-8af2-9edbe3407f28'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:44.847772 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-f20a5788-7ba6-49ae-8af2-9edbe3407f28 admin admin] The token cache is being invalidate because project d925def3f55b435bb5452a0b5e5daff5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81341) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:59:44.848081 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-f20a5788-7ba6-49ae-8af2-9edbe3407f28 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:44.848707 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-f20a5788-7ba6-49ae-8af2-9edbe3407f28 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:44.850176 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1018/2042] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Sep 1 14:59:44 2022] DELETE /identity/v3/projects/d925def3f55b435bb5452a0b5e5daff5 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:59:44.855118 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec84bbd6-dc26-4cd4-a579-5d47be4f262c None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:44.855427 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec84bbd6-dc26-4cd4-a579-5d47be4f262c None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:44.855693 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec84bbd6-dc26-4cd4-a579-5d47be4f262c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:44.875094 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-ec84bbd6-dc26-4cd4-a579-5d47be4f262c None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:59:44.876282 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-ec84bbd6-dc26-4cd4-a579-5d47be4f262c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:59:44.914756 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:59:44.934586 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1025/2043] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 14:59:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 14:59:44.942226 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd13777b-b42f-41a4-8efe-bca90629f70e None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:44.942536 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd13777b-b42f-41a4-8efe-bca90629f70e None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:44.942806 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd13777b-b42f-41a4-8efe-bca90629f70e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:44.965626 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-bd13777b-b42f-41a4-8efe-bca90629f70e None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:59:44.967001 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-bd13777b-b42f-41a4-8efe-bca90629f70e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:59:44.975393 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1019/2044] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 14:59:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:59:44.979546 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec84bbd6-dc26-4cd4-a579-5d47be4f262c None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:59:44.989907 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec84bbd6-dc26-4cd4-a579-5d47be4f262c None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:59:44.994422 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7447ef-5b13-4a9a-9bf9-02cd824328dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:59:44.995010 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7447ef-5b13-4a9a-9bf9-02cd824328dd admin admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:44.995287 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7447ef-5b13-4a9a-9bf9-02cd824328dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:44.995569 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7447ef-5b13-4a9a-9bf9-02cd824328dd admin admin] PATH_INFO: `/v3/domains` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:44.996193 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7447ef-5b13-4a9a-9bf9-02cd824328dd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:59:44.996487 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7447ef-5b13-4a9a-9bf9-02cd824328dd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:59:44.998068 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7447ef-5b13-4a9a-9bf9-02cd824328dd admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:59:45.009384 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1026/2045] 10.222.0.33 () {58 vars in 1153 bytes} [Thu Sep 1 14:59:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:59:45.013348 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd13777b-b42f-41a4-8efe-bca90629f70e None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:59:45.022600 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd13777b-b42f-41a4-8efe-bca90629f70e None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:59:45.027180 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fdccc5-eaad-4e7b-972d-013a17d40ec2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:59:45.027762 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fdccc5-eaad-4e7b-972d-013a17d40ec2 admin admin] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:45.028040 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fdccc5-eaad-4e7b-972d-013a17d40ec2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:45.028309 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fdccc5-eaad-4e7b-972d-013a17d40ec2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:45.028933 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fdccc5-eaad-4e7b-972d-013a17d40ec2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:59:45.030877 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fdccc5-eaad-4e7b-972d-013a17d40ec2 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:59:45.057779 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1020/2046] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Sep 1 14:59:45 2022] POST /identity/v3/projects => generated 383 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:59:45.061873 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7447ef-5b13-4a9a-9bf9-02cd824328dd admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:59:45.071629 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7447ef-5b13-4a9a-9bf9-02cd824328dd admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:59:45.128274 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd619fa-9bd0-4e1f-885f-1889843fe347 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:59:45.128883 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd619fa-9bd0-4e1f-885f-1889843fe347 admin admin] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:45.129161 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd619fa-9bd0-4e1f-885f-1889843fe347 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:45.129432 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd619fa-9bd0-4e1f-885f-1889843fe347 admin admin] PATH_INFO: `/v3/users` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:45.130063 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd619fa-9bd0-4e1f-885f-1889843fe347 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:59:45.132040 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd619fa-9bd0-4e1f-885f-1889843fe347 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:59:45.153078 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1027/2047] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Sep 1 14:59:45 2022] POST /identity/v3/users => generated 292 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 14:59:45.157174 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fdccc5-eaad-4e7b-972d-013a17d40ec2 admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:59:45.167952 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fdccc5-eaad-4e7b-972d-013a17d40ec2 admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:59:45.172489 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb917fc-ef45-4524-b619-dcf43e87b384 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:59:45.173071 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb917fc-ef45-4524-b619-dcf43e87b384 admin admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:45.173351 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb917fc-ef45-4524-b619-dcf43e87b384 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:45.173620 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb917fc-ef45-4524-b619-dcf43e87b384 admin admin] PATH_INFO: `/v3/roles` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:45.174323 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb917fc-ef45-4524-b619-dcf43e87b384 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:59:45.175927 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb917fc-ef45-4524-b619-dcf43e87b384 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:59:45.185837 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1021/2048] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 1 14:59:45 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:59:45.189692 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd619fa-9bd0-4e1f-885f-1889843fe347 admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:59:45.199657 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd619fa-9bd0-4e1f-885f-1889843fe347 admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:59:45.204570 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e22f85-a5f6-4c95-911f-3187e608ad50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:59:45.205168 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e22f85-a5f6-4c95-911f-3187e608ad50 admin admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:45.205953 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e22f85-a5f6-4c95-911f-3187e608ad50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:45.206197 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e22f85-a5f6-4c95-911f-3187e608ad50 admin admin] PATH_INFO: `/v3/roles` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:45.206910 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e22f85-a5f6-4c95-911f-3187e608ad50 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:59:45.208791 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e22f85-a5f6-4c95-911f-3187e608ad50 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:59:45.220049 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1028/2049] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 1 14:59:45 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:59:45.224885 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb917fc-ef45-4524-b619-dcf43e87b384 admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:59:45.234976 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb917fc-ef45-4524-b619-dcf43e87b384 admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:59:45.239705 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1947ee47-bb6b-4625-b022-64be60ed1b30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:59:45.240353 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1947ee47-bb6b-4625-b022-64be60ed1b30 admin admin] REQUEST_METHOD: `PUT` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:45.240633 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1947ee47-bb6b-4625-b022-64be60ed1b30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:45.240909 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1947ee47-bb6b-4625-b022-64be60ed1b30 admin admin] PATH_INFO: `/v3/projects/96a9dae3808140258407c30997c62b73/users/aae2f6b4ac014595a04b1e136ccf13b0/roles/5d1feef0e13d4874928bd86258c4e3b2` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:45.255903 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1947ee47-bb6b-4625-b022-64be60ed1b30 admin admin] RBAC: Authorizing `identity:create_grant(project_id=96a9dae3808140258407c30997c62b73, user_id=aae2f6b4ac014595a04b1e136ccf13b0, role_id=5d1feef0e13d4874928bd86258c4e3b2)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:59:45.258637 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1947ee47-bb6b-4625-b022-64be60ed1b30 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:59:45.271556 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1022/2050] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 1 14:59:45 2022] PUT /identity/v3/projects/96a9dae3808140258407c30997c62b73/users/aae2f6b4ac014595a04b1e136ccf13b0/roles/5d1feef0e13d4874928bd86258c4e3b2 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:59:45.281720 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7dc6c7-2322-401d-a2d8-5e2759f63cff None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:45.282323 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7dc6c7-2322-401d-a2d8-5e2759f63cff None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:45.282323 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7dc6c7-2322-401d-a2d8-5e2759f63cff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:45.310662 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-4b7dc6c7-2322-401d-a2d8-5e2759f63cff None None] MFA Rules not processed for user `aae2f6b4ac014595a04b1e136ccf13b0`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:59:45.311894 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-4b7dc6c7-2322-401d-a2d8-5e2759f63cff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:59:45.374342 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1029/2051] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 14:59:45 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:59:45.384780 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1947ee47-bb6b-4625-b022-64be60ed1b30 admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:59:45.386729 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-1947ee47-bb6b-4625-b022-64be60ed1b30 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:59:45.468003 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1947ee47-bb6b-4625-b022-64be60ed1b30 admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:59:45.473964 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd6ef4e-1f16-46f1-bc96-78bb7ef999e4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4858bb8621854ec8aba836e7d9050498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:59:45.474614 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd6ef4e-1f16-46f1-bc96-78bb7ef999e4 service nova] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:45.474982 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd6ef4e-1f16-46f1-bc96-78bb7ef999e4 service nova] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:45.475270 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd6ef4e-1f16-46f1-bc96-78bb7ef999e4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:45.484341 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd6ef4e-1f16-46f1-bc96-78bb7ef999e4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:59:45.486152 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd6ef4e-1f16-46f1-bc96-78bb7ef999e4 service nova] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:59:45.489996 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:59:45.496280 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1023/2052] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 14:59:45 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:59:47.729734 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7dc6c7-2322-401d-a2d8-5e2759f63cff None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:59:47.731387 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-4b7dc6c7-2322-401d-a2d8-5e2759f63cff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:59:47.777380 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:59:47.807501 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7dc6c7-2322-401d-a2d8-5e2759f63cff None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:59:47.812340 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383c204c-2313-49b6-a126-035c4849ccd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:59:47.813017 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383c204c-2313-49b6-a126-035c4849ccd0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:47.813492 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383c204c-2313-49b6-a126-035c4849ccd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:47.813492 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383c204c-2313-49b6-a126-035c4849ccd0 admin admin] PATH_INFO: `/v3/users/ddf627f6fa19409298cf50aa8d89b4b5` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:47.814766 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383c204c-2313-49b6-a126-035c4849ccd0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ddf627f6fa19409298cf50aa8d89b4b5)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:59:47.816685 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383c204c-2313-49b6-a126-035c4849ccd0 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:59:47.870737 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-383c204c-2313-49b6-a126-035c4849ccd0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ddf627f6fa19409298cf50aa8d89b4b5'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:47.879832 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-383c204c-2313-49b6-a126-035c4849ccd0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ddf627f6fa19409298cf50aa8d89b4b5'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:47.880471 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-383c204c-2313-49b6-a126-035c4849ccd0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ddf627f6fa19409298cf50aa8d89b4b5'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:47.892345 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-383c204c-2313-49b6-a126-035c4849ccd0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ddf627f6fa19409298cf50aa8d89b4b5'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:47.893147 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-383c204c-2313-49b6-a126-035c4849ccd0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ddf627f6fa19409298cf50aa8d89b4b5'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:47.905154 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1030/2053] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Sep 1 14:59:47 2022] DELETE /identity/v3/users/ddf627f6fa19409298cf50aa8d89b4b5 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:59:47.910316 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ab3d98-6fcf-4677-a9fa-d7ebbd7edd98 None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:47.910801 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ab3d98-6fcf-4677-a9fa-d7ebbd7edd98 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:47.910801 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ab3d98-6fcf-4677-a9fa-d7ebbd7edd98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:47.931542 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-b4ab3d98-6fcf-4677-a9fa-d7ebbd7edd98 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:59:47.932845 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-b4ab3d98-6fcf-4677-a9fa-d7ebbd7edd98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:59:48.004530 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1024/2054] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 14:59:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:59:48.015889 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383c204c-2313-49b6-a126-035c4849ccd0 admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:59:48.017516 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-383c204c-2313-49b6-a126-035c4849ccd0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:59:48.098098 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383c204c-2313-49b6-a126-035c4849ccd0 admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:59:48.102925 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a195a2-0d48-41a5-8c83-2a1fe0700e41 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2790149639aa418bbc4e4982e93eda44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:59:48.103490 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a195a2-0d48-41a5-8c83-2a1fe0700e41 service neutron] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:48.103839 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a195a2-0d48-41a5-8c83-2a1fe0700e41 service neutron] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:48.104688 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a195a2-0d48-41a5-8c83-2a1fe0700e41 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:48.125185 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a195a2-0d48-41a5-8c83-2a1fe0700e41 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:59:48.126850 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a195a2-0d48-41a5-8c83-2a1fe0700e41 service neutron] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:59:48.137730 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1031/2055] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 14:59:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:59:48.607260 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ab3d98-6fcf-4677-a9fa-d7ebbd7edd98 None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:59:48.608727 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-b4ab3d98-6fcf-4677-a9fa-d7ebbd7edd98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:59:48.623376 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ab3d98-6fcf-4677-a9fa-d7ebbd7edd98 None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:59:48.628475 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385cb686-63ba-4796-aba9-dbc8d4f80acb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:59:48.629309 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385cb686-63ba-4796-aba9-dbc8d4f80acb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:48.629680 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385cb686-63ba-4796-aba9-dbc8d4f80acb admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:48.629834 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385cb686-63ba-4796-aba9-dbc8d4f80acb admin admin] PATH_INFO: `/v3/projects/2684616daed843f09428fdc619aa934f` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:48.630978 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385cb686-63ba-4796-aba9-dbc8d4f80acb admin admin] RBAC: Authorizing `identity:delete_project(project_id=2684616daed843f09428fdc619aa934f)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:59:48.632888 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385cb686-63ba-4796-aba9-dbc8d4f80acb admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:59:48.680617 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-385cb686-63ba-4796-aba9-dbc8d4f80acb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2684616daed843f09428fdc619aa934f', 'request_id': 'req-385cb686-63ba-4796-aba9-dbc8d4f80acb'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:48.686748 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-385cb686-63ba-4796-aba9-dbc8d4f80acb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2684616daed843f09428fdc619aa934f', 'request_id': 'req-385cb686-63ba-4796-aba9-dbc8d4f80acb'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:48.694423 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-385cb686-63ba-4796-aba9-dbc8d4f80acb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2684616daed843f09428fdc619aa934f', 'request_id': 'req-385cb686-63ba-4796-aba9-dbc8d4f80acb'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:48.704458 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-385cb686-63ba-4796-aba9-dbc8d4f80acb admin admin] The token cache is being invalidate because project 2684616daed843f09428fdc619aa934f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81341) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:59:48.704766 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-385cb686-63ba-4796-aba9-dbc8d4f80acb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:48.705379 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-385cb686-63ba-4796-aba9-dbc8d4f80acb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:48.706915 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1025/2056] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Sep 1 14:59:48 2022] DELETE /identity/v3/projects/2684616daed843f09428fdc619aa934f => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:59:48.711096 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a195a2-0d48-41a5-8c83-2a1fe0700e41 service neutron] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:59:48.720589 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a195a2-0d48-41a5-8c83-2a1fe0700e41 service neutron] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:59:48.775557 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c512402-272f-4576-befb-b29edb17a99f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:59:48.776173 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c512402-272f-4576-befb-b29edb17a99f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:48.776434 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c512402-272f-4576-befb-b29edb17a99f admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:48.776700 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c512402-272f-4576-befb-b29edb17a99f admin admin] PATH_INFO: `/v3/users/34c146d16faa4ddcbb1956535303214a` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:48.777799 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c512402-272f-4576-befb-b29edb17a99f admin admin] RBAC: Authorizing `identity:delete_user(user_id=34c146d16faa4ddcbb1956535303214a)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:59:48.779783 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c512402-272f-4576-befb-b29edb17a99f admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:59:48.781960 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:59:48.828679 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-5c512402-272f-4576-befb-b29edb17a99f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '34c146d16faa4ddcbb1956535303214a'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:48.836152 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-5c512402-272f-4576-befb-b29edb17a99f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '34c146d16faa4ddcbb1956535303214a'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:48.836747 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-5c512402-272f-4576-befb-b29edb17a99f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '34c146d16faa4ddcbb1956535303214a'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:48.848896 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-5c512402-272f-4576-befb-b29edb17a99f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '34c146d16faa4ddcbb1956535303214a'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:48.849518 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-5c512402-272f-4576-befb-b29edb17a99f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '34c146d16faa4ddcbb1956535303214a'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:48.861328 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1032/2057] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Sep 1 14:59:48 2022] DELETE /identity/v3/users/34c146d16faa4ddcbb1956535303214a => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:59:49.140897 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385cb686-63ba-4796-aba9-dbc8d4f80acb admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:59:49.143657 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-385cb686-63ba-4796-aba9-dbc8d4f80acb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:59:49.159497 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385cb686-63ba-4796-aba9-dbc8d4f80acb admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:59:49.219366 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e299e50-5ee1-45a0-a5cd-78422ac83a0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:59:49.220091 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e299e50-5ee1-45a0-a5cd-78422ac83a0f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:49.220194 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e299e50-5ee1-45a0-a5cd-78422ac83a0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:49.220427 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e299e50-5ee1-45a0-a5cd-78422ac83a0f admin admin] PATH_INFO: `/v3/projects/a5897696fdbf4cd7a72330f7327c85dc` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:49.221620 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e299e50-5ee1-45a0-a5cd-78422ac83a0f admin admin] RBAC: Authorizing `identity:delete_project(project_id=a5897696fdbf4cd7a72330f7327c85dc)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:59:49.223548 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e299e50-5ee1-45a0-a5cd-78422ac83a0f admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:59:49.247993 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:59:49.274383 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-5e299e50-5ee1-45a0-a5cd-78422ac83a0f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a5897696fdbf4cd7a72330f7327c85dc', 'request_id': 'req-5e299e50-5ee1-45a0-a5cd-78422ac83a0f'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:49.280537 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-5e299e50-5ee1-45a0-a5cd-78422ac83a0f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a5897696fdbf4cd7a72330f7327c85dc', 'request_id': 'req-5e299e50-5ee1-45a0-a5cd-78422ac83a0f'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:49.288094 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-5e299e50-5ee1-45a0-a5cd-78422ac83a0f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a5897696fdbf4cd7a72330f7327c85dc', 'request_id': 'req-5e299e50-5ee1-45a0-a5cd-78422ac83a0f'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:49.298681 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-5e299e50-5ee1-45a0-a5cd-78422ac83a0f admin admin] The token cache is being invalidate because project a5897696fdbf4cd7a72330f7327c85dc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81341) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:59:49.299042 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-5e299e50-5ee1-45a0-a5cd-78422ac83a0f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:49.299789 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-5e299e50-5ee1-45a0-a5cd-78422ac83a0f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:49.301349 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1026/2058] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Sep 1 14:59:49 2022] DELETE /identity/v3/projects/a5897696fdbf4cd7a72330f7327c85dc => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:59:49.305587 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c512402-272f-4576-befb-b29edb17a99f admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:59:49.307171 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-5c512402-272f-4576-befb-b29edb17a99f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:59:49.321193 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c512402-272f-4576-befb-b29edb17a99f admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:59:49.379937 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba9f379-6f18-4436-b352-9d212e15e85e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:59:49.380657 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba9f379-6f18-4436-b352-9d212e15e85e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:49.381022 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba9f379-6f18-4436-b352-9d212e15e85e admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:49.381362 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba9f379-6f18-4436-b352-9d212e15e85e admin admin] PATH_INFO: `/v3/users/62d6b8950b26429d83eff77c2edaf1d1` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:49.382584 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba9f379-6f18-4436-b352-9d212e15e85e admin admin] RBAC: Authorizing `identity:delete_user(user_id=62d6b8950b26429d83eff77c2edaf1d1)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:59:49.384563 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba9f379-6f18-4436-b352-9d212e15e85e admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:59:49.442514 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-aba9f379-6f18-4436-b352-9d212e15e85e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '62d6b8950b26429d83eff77c2edaf1d1'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:49.449606 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-aba9f379-6f18-4436-b352-9d212e15e85e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '62d6b8950b26429d83eff77c2edaf1d1'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:49.450263 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-aba9f379-6f18-4436-b352-9d212e15e85e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '62d6b8950b26429d83eff77c2edaf1d1'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:49.462805 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-aba9f379-6f18-4436-b352-9d212e15e85e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '62d6b8950b26429d83eff77c2edaf1d1'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:49.463439 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-aba9f379-6f18-4436-b352-9d212e15e85e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '62d6b8950b26429d83eff77c2edaf1d1'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:49.471776 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:59:49.476666 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1033/2059] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Sep 1 14:59:49 2022] DELETE /identity/v3/users/62d6b8950b26429d83eff77c2edaf1d1 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:59:49.759641 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e299e50-5ee1-45a0-a5cd-78422ac83a0f admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:59:49.761446 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-5e299e50-5ee1-45a0-a5cd-78422ac83a0f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:59:49.774997 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e299e50-5ee1-45a0-a5cd-78422ac83a0f admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:59:49.832880 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3ebd2c-696e-4cb7-99ae-bb997acc4cea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:59:49.833512 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3ebd2c-696e-4cb7-99ae-bb997acc4cea admin admin] REQUEST_METHOD: `DELETE` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:49.833797 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3ebd2c-696e-4cb7-99ae-bb997acc4cea admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:49.834063 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3ebd2c-696e-4cb7-99ae-bb997acc4cea admin admin] PATH_INFO: `/v3/projects/1fa7856e698843d1917437f0a2cbbeb2` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:49.835230 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3ebd2c-696e-4cb7-99ae-bb997acc4cea admin admin] RBAC: Authorizing `identity:delete_project(project_id=1fa7856e698843d1917437f0a2cbbeb2)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:59:49.837155 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3ebd2c-696e-4cb7-99ae-bb997acc4cea admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:59:49.883392 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-2c3ebd2c-696e-4cb7-99ae-bb997acc4cea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1fa7856e698843d1917437f0a2cbbeb2', 'request_id': 'req-2c3ebd2c-696e-4cb7-99ae-bb997acc4cea'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:49.889781 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-2c3ebd2c-696e-4cb7-99ae-bb997acc4cea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1fa7856e698843d1917437f0a2cbbeb2', 'request_id': 'req-2c3ebd2c-696e-4cb7-99ae-bb997acc4cea'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:49.898263 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-2c3ebd2c-696e-4cb7-99ae-bb997acc4cea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1fa7856e698843d1917437f0a2cbbeb2', 'request_id': 'req-2c3ebd2c-696e-4cb7-99ae-bb997acc4cea'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:49.909015 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-2c3ebd2c-696e-4cb7-99ae-bb997acc4cea admin admin] The token cache is being invalidate because project 1fa7856e698843d1917437f0a2cbbeb2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81341) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:59:49.909293 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-2c3ebd2c-696e-4cb7-99ae-bb997acc4cea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:49.909902 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-2c3ebd2c-696e-4cb7-99ae-bb997acc4cea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:49.911396 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1027/2060] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Sep 1 14:59:49 2022] DELETE /identity/v3/projects/1fa7856e698843d1917437f0a2cbbeb2 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:59:51.258455 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba9f379-6f18-4436-b352-9d212e15e85e admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:59:51.260209 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-aba9f379-6f18-4436-b352-9d212e15e85e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:59:51.273726 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba9f379-6f18-4436-b352-9d212e15e85e admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:59:51.328551 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acabc326-3d9d-4dbc-8f25-20ab904c2b16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:59:51.329196 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acabc326-3d9d-4dbc-8f25-20ab904c2b16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:51.329475 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acabc326-3d9d-4dbc-8f25-20ab904c2b16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:51.329751 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acabc326-3d9d-4dbc-8f25-20ab904c2b16 admin admin] PATH_INFO: `/v3/users/53a257a41bbb4888901442650d4ba5a7` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:51.331442 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acabc326-3d9d-4dbc-8f25-20ab904c2b16 admin admin] RBAC: Authorizing `identity:delete_user(user_id=53a257a41bbb4888901442650d4ba5a7)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:59:51.333372 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acabc326-3d9d-4dbc-8f25-20ab904c2b16 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:59:51.379907 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-acabc326-3d9d-4dbc-8f25-20ab904c2b16 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '53a257a41bbb4888901442650d4ba5a7'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:51.391473 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-acabc326-3d9d-4dbc-8f25-20ab904c2b16 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '53a257a41bbb4888901442650d4ba5a7'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:51.392087 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-acabc326-3d9d-4dbc-8f25-20ab904c2b16 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '53a257a41bbb4888901442650d4ba5a7'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:51.404451 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-acabc326-3d9d-4dbc-8f25-20ab904c2b16 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '53a257a41bbb4888901442650d4ba5a7'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:51.405072 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-acabc326-3d9d-4dbc-8f25-20ab904c2b16 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '53a257a41bbb4888901442650d4ba5a7'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:51.417905 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1034/2061] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Sep 1 14:59:51 2022] DELETE /identity/v3/users/53a257a41bbb4888901442650d4ba5a7 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:59:51.422522 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fec6fbd-e12c-488a-b39a-a2f03918e886 None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:51.422973 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fec6fbd-e12c-488a-b39a-a2f03918e886 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:51.423083 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fec6fbd-e12c-488a-b39a-a2f03918e886 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:51.442567 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-0fec6fbd-e12c-488a-b39a-a2f03918e886 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:59:51.443771 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-0fec6fbd-e12c-488a-b39a-a2f03918e886 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:59:51.484956 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:59:51.514110 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1028/2062] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 14:59:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:59:51.525139 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acabc326-3d9d-4dbc-8f25-20ab904c2b16 admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:59:51.526939 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-acabc326-3d9d-4dbc-8f25-20ab904c2b16 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:59:51.591192 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acabc326-3d9d-4dbc-8f25-20ab904c2b16 admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:59:51.594985 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68a93ed-1350-4f9b-b676-263f28b9c693 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2790149639aa418bbc4e4982e93eda44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:59:51.595623 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68a93ed-1350-4f9b-b676-263f28b9c693 service neutron] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:51.595918 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68a93ed-1350-4f9b-b676-263f28b9c693 service neutron] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:51.596185 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68a93ed-1350-4f9b-b676-263f28b9c693 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:51.606089 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68a93ed-1350-4f9b-b676-263f28b9c693 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:59:51.607776 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68a93ed-1350-4f9b-b676-263f28b9c693 service neutron] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:59:51.622288 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1035/2063] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 14:59:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:59:51.905217 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fec6fbd-e12c-488a-b39a-a2f03918e886 None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:59:51.906824 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-0fec6fbd-e12c-488a-b39a-a2f03918e886 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:59:51.918064 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68a93ed-1350-4f9b-b676-263f28b9c693 service neutron] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:59:51.919608 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fec6fbd-e12c-488a-b39a-a2f03918e886 None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:59:51.919724 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-c68a93ed-1350-4f9b-b676-263f28b9c693 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:59:51.932946 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c225e8fa-421d-47e0-bc55-7be9552984bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:59:51.933608 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c225e8fa-421d-47e0-bc55-7be9552984bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:51.933885 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c225e8fa-421d-47e0-bc55-7be9552984bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:51.934152 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c225e8fa-421d-47e0-bc55-7be9552984bd admin admin] PATH_INFO: `/v3/users/619b17c6d75749eba9fbfda01d26c292` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:51.935460 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c225e8fa-421d-47e0-bc55-7be9552984bd admin admin] RBAC: Authorizing `identity:delete_user(user_id=619b17c6d75749eba9fbfda01d26c292)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:59:51.937318 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c225e8fa-421d-47e0-bc55-7be9552984bd admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:59:51.937871 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68a93ed-1350-4f9b-b676-263f28b9c693 service neutron] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:59:51.945688 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bc7394-6ad8-4f75-a736-9dcfab6101cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:59:51.946357 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bc7394-6ad8-4f75-a736-9dcfab6101cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:51.946641 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bc7394-6ad8-4f75-a736-9dcfab6101cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:51.946915 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bc7394-6ad8-4f75-a736-9dcfab6101cc admin admin] PATH_INFO: `/v3/projects/470ea9bada8a49c9840e745de1c91f28` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:51.948142 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bc7394-6ad8-4f75-a736-9dcfab6101cc admin admin] RBAC: Authorizing `identity:delete_project(project_id=470ea9bada8a49c9840e745de1c91f28)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:59:51.950397 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bc7394-6ad8-4f75-a736-9dcfab6101cc admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:59:51.972646 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:59:51.990146 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-c225e8fa-421d-47e0-bc55-7be9552984bd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '619b17c6d75749eba9fbfda01d26c292'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:51.997621 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-c225e8fa-421d-47e0-bc55-7be9552984bd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '619b17c6d75749eba9fbfda01d26c292'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:51.998324 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-c225e8fa-421d-47e0-bc55-7be9552984bd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '619b17c6d75749eba9fbfda01d26c292'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:52.005926 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-c6bc7394-6ad8-4f75-a736-9dcfab6101cc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '470ea9bada8a49c9840e745de1c91f28', 'request_id': 'req-c6bc7394-6ad8-4f75-a736-9dcfab6101cc'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:52.011777 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-c225e8fa-421d-47e0-bc55-7be9552984bd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '619b17c6d75749eba9fbfda01d26c292'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:52.012425 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-c225e8fa-421d-47e0-bc55-7be9552984bd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '619b17c6d75749eba9fbfda01d26c292'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:52.012562 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-c6bc7394-6ad8-4f75-a736-9dcfab6101cc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '470ea9bada8a49c9840e745de1c91f28', 'request_id': 'req-c6bc7394-6ad8-4f75-a736-9dcfab6101cc'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:52.020007 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-c6bc7394-6ad8-4f75-a736-9dcfab6101cc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '470ea9bada8a49c9840e745de1c91f28', 'request_id': 'req-c6bc7394-6ad8-4f75-a736-9dcfab6101cc'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:52.025005 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1029/2064] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Sep 1 14:59:51 2022] DELETE /identity/v3/users/619b17c6d75749eba9fbfda01d26c292 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:59:52.029573 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9af2b3-632b-4628-8e9c-58fd6140b1f6 None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:52.030006 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9af2b3-632b-4628-8e9c-58fd6140b1f6 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:52.030311 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9af2b3-632b-4628-8e9c-58fd6140b1f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:52.031380 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-c6bc7394-6ad8-4f75-a736-9dcfab6101cc admin admin] The token cache is being invalidate because project 470ea9bada8a49c9840e745de1c91f28 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81342) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:59:52.031679 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-c6bc7394-6ad8-4f75-a736-9dcfab6101cc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:52.032337 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-c6bc7394-6ad8-4f75-a736-9dcfab6101cc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:52.034095 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1036/2065] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Sep 1 14:59:51 2022] DELETE /identity/v3/projects/470ea9bada8a49c9840e745de1c91f28 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 14:59:52.039034 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f90e434-bb2e-4c21-b97d-24ee8d0938ea None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:52.039428 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f90e434-bb2e-4c21-b97d-24ee8d0938ea None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:52.039782 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f90e434-bb2e-4c21-b97d-24ee8d0938ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:52.050864 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-4d9af2b3-632b-4628-8e9c-58fd6140b1f6 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:59:52.052065 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-4d9af2b3-632b-4628-8e9c-58fd6140b1f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:59:52.060257 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-1f90e434-bb2e-4c21-b97d-24ee8d0938ea None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:59:52.061759 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-1f90e434-bb2e-4c21-b97d-24ee8d0938ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:59:52.116856 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1030/2066] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 14:59:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 14:59:52.128526 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9af2b3-632b-4628-8e9c-58fd6140b1f6 None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:59:52.129979 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-4d9af2b3-632b-4628-8e9c-58fd6140b1f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:59:52.132546 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1037/2067] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 14:59:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 14:59:52.141316 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80760a7c-e939-4c27-9471-2cddd9b16e6a None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:52.141602 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80760a7c-e939-4c27-9471-2cddd9b16e6a None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:52.141879 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80760a7c-e939-4c27-9471-2cddd9b16e6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:52.161852 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-80760a7c-e939-4c27-9471-2cddd9b16e6a None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:59:52.163111 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-80760a7c-e939-4c27-9471-2cddd9b16e6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:59:52.171153 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1038/2068] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 14:59:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 14:59:52.175956 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80760a7c-e939-4c27-9471-2cddd9b16e6a None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:59:52.186453 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80760a7c-e939-4c27-9471-2cddd9b16e6a None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:59:52.191653 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f63f74-2e4f-4b19-abf6-c21fc748db01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:59:52.192259 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f63f74-2e4f-4b19-abf6-c21fc748db01 admin admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:52.192539 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f63f74-2e4f-4b19-abf6-c21fc748db01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:52.192808 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f63f74-2e4f-4b19-abf6-c21fc748db01 admin admin] PATH_INFO: `/v3/domains` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:52.193307 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9af2b3-632b-4628-8e9c-58fd6140b1f6 None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:59:52.193461 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f63f74-2e4f-4b19-abf6-c21fc748db01 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:59:52.193763 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f63f74-2e4f-4b19-abf6-c21fc748db01 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:59:52.195401 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f63f74-2e4f-4b19-abf6-c21fc748db01 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:59:52.197185 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a44ad8d-7a2c-4af4-a872-1712fe5a0381 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2790149639aa418bbc4e4982e93eda44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:59:52.197785 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a44ad8d-7a2c-4af4-a872-1712fe5a0381 service neutron] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:52.198062 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a44ad8d-7a2c-4af4-a872-1712fe5a0381 service neutron] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:52.198354 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a44ad8d-7a2c-4af4-a872-1712fe5a0381 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:52.207019 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1039/2069] 10.222.0.33 () {58 vars in 1153 bytes} [Thu Sep 1 14:59:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 14:59:52.207587 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a44ad8d-7a2c-4af4-a872-1712fe5a0381 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:59:52.209158 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a44ad8d-7a2c-4af4-a872-1712fe5a0381 service neutron] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:59:52.211500 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f63f74-2e4f-4b19-abf6-c21fc748db01 admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:59:52.213146 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:59:52.223992 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1031/2070] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 14:59:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 14:59:52.224438 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f63f74-2e4f-4b19-abf6-c21fc748db01 admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:59:52.229704 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6305ce-a095-4931-8598-4a225e8619fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:59:52.230477 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6305ce-a095-4931-8598-4a225e8619fd admin admin] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:52.230771 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6305ce-a095-4931-8598-4a225e8619fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:52.233497 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6305ce-a095-4931-8598-4a225e8619fd admin admin] PATH_INFO: `/v3/projects` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:52.234165 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6305ce-a095-4931-8598-4a225e8619fd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:59:52.236115 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6305ce-a095-4931-8598-4a225e8619fd admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:59:52.268754 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1040/2071] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Sep 1 14:59:52 2022] POST /identity/v3/projects => generated 375 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 14:59:52.273106 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a44ad8d-7a2c-4af4-a872-1712fe5a0381 service neutron] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:59:52.286723 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a44ad8d-7a2c-4af4-a872-1712fe5a0381 service neutron] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:59:52.350136 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b230b3-8a51-4854-aed9-b19146167846 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:59:52.350771 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b230b3-8a51-4854-aed9-b19146167846 admin admin] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:52.351051 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b230b3-8a51-4854-aed9-b19146167846 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:52.351325 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b230b3-8a51-4854-aed9-b19146167846 admin admin] PATH_INFO: `/v3/users` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:52.351960 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b230b3-8a51-4854-aed9-b19146167846 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:59:52.353875 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b230b3-8a51-4854-aed9-b19146167846 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:59:52.384109 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1032/2072] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Sep 1 14:59:52 2022] POST /identity/v3/users => generated 288 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:59:52.387959 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6305ce-a095-4931-8598-4a225e8619fd admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:59:52.398551 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6305ce-a095-4931-8598-4a225e8619fd admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:59:52.403249 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ad9e5f-467b-4aa1-b5cd-2e1a3457601b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:59:52.403848 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ad9e5f-467b-4aa1-b5cd-2e1a3457601b admin admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:52.404129 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ad9e5f-467b-4aa1-b5cd-2e1a3457601b admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:52.404400 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ad9e5f-467b-4aa1-b5cd-2e1a3457601b admin admin] PATH_INFO: `/v3/roles` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:52.405042 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ad9e5f-467b-4aa1-b5cd-2e1a3457601b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:59:52.406679 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ad9e5f-467b-4aa1-b5cd-2e1a3457601b admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:59:52.420114 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1041/2073] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 1 14:59:52 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 14:59:52.421271 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b230b3-8a51-4854-aed9-b19146167846 admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:59:52.432221 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b230b3-8a51-4854-aed9-b19146167846 admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:59:52.436990 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80903a7-9531-41e1-b610-380fc95cdc19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:59:52.437586 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80903a7-9531-41e1-b610-380fc95cdc19 admin admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:52.437889 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80903a7-9531-41e1-b610-380fc95cdc19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:52.438166 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80903a7-9531-41e1-b610-380fc95cdc19 admin admin] PATH_INFO: `/v3/roles` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:52.438839 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80903a7-9531-41e1-b610-380fc95cdc19 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:59:52.440719 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80903a7-9531-41e1-b610-380fc95cdc19 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:59:52.455566 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1033/2074] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 1 14:59:52 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:59:52.457753 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ad9e5f-467b-4aa1-b5cd-2e1a3457601b admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:59:52.467800 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ad9e5f-467b-4aa1-b5cd-2e1a3457601b admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:59:52.472980 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6b3d55-5bad-42d1-80ff-5c8b4b4177f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:59:52.473631 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6b3d55-5bad-42d1-80ff-5c8b4b4177f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:52.473908 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6b3d55-5bad-42d1-80ff-5c8b4b4177f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:52.474178 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6b3d55-5bad-42d1-80ff-5c8b4b4177f2 admin admin] PATH_INFO: `/v3/projects/e70768e1f7f149578261a30ec295d59d/users/5710749db2ed489fb3ce3b4cc8c9cf8a/roles/5d1feef0e13d4874928bd86258c4e3b2` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:52.490269 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6b3d55-5bad-42d1-80ff-5c8b4b4177f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e70768e1f7f149578261a30ec295d59d, user_id=5710749db2ed489fb3ce3b4cc8c9cf8a, role_id=5d1feef0e13d4874928bd86258c4e3b2)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:59:52.492894 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6b3d55-5bad-42d1-80ff-5c8b4b4177f2 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:59:52.506461 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1042/2075] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 1 14:59:52 2022] PUT /identity/v3/projects/e70768e1f7f149578261a30ec295d59d/users/5710749db2ed489fb3ce3b4cc8c9cf8a/roles/5d1feef0e13d4874928bd86258c4e3b2 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 14:59:52.517483 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b783b338-152c-4821-9319-dbea19e5a825 None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:52.517770 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b783b338-152c-4821-9319-dbea19e5a825 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:52.518043 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b783b338-152c-4821-9319-dbea19e5a825 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:52.519321 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6b3d55-5bad-42d1-80ff-5c8b4b4177f2 admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:59:52.521303 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-5d6b3d55-5bad-42d1-80ff-5c8b4b4177f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:59:52.553931 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-b783b338-152c-4821-9319-dbea19e5a825 None None] MFA Rules not processed for user `5710749db2ed489fb3ce3b4cc8c9cf8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:59:52.555530 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-b783b338-152c-4821-9319-dbea19e5a825 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:59:52.565156 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:59:52.602780 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6b3d55-5bad-42d1-80ff-5c8b4b4177f2 admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:59:52.607991 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb06374c-3784-4edb-a72f-2e1d833cc8eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:59:52.608646 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb06374c-3784-4edb-a72f-2e1d833cc8eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:52.608976 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb06374c-3784-4edb-a72f-2e1d833cc8eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:52.609233 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb06374c-3784-4edb-a72f-2e1d833cc8eb admin admin] PATH_INFO: `/v3/projects/baf1a0cc82104a8ea760e2e861bf4036` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:52.610483 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb06374c-3784-4edb-a72f-2e1d833cc8eb admin admin] RBAC: Authorizing `identity:delete_project(project_id=baf1a0cc82104a8ea760e2e861bf4036)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:59:52.612516 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb06374c-3784-4edb-a72f-2e1d833cc8eb admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:59:52.622579 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1034/2076] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 14:59:52 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:59:52.634056 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b783b338-152c-4821-9319-dbea19e5a825 None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:59:52.639973 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-b783b338-152c-4821-9319-dbea19e5a825 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:59:52.680544 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-eb06374c-3784-4edb-a72f-2e1d833cc8eb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'baf1a0cc82104a8ea760e2e861bf4036', 'request_id': 'req-eb06374c-3784-4edb-a72f-2e1d833cc8eb'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:52.699168 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-eb06374c-3784-4edb-a72f-2e1d833cc8eb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'baf1a0cc82104a8ea760e2e861bf4036', 'request_id': 'req-eb06374c-3784-4edb-a72f-2e1d833cc8eb'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:52.704767 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b783b338-152c-4821-9319-dbea19e5a825 None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:59:52.708411 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-eb06374c-3784-4edb-a72f-2e1d833cc8eb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'baf1a0cc82104a8ea760e2e861bf4036', 'request_id': 'req-eb06374c-3784-4edb-a72f-2e1d833cc8eb'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:52.727401 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-eb06374c-3784-4edb-a72f-2e1d833cc8eb admin admin] The token cache is being invalidate because project baf1a0cc82104a8ea760e2e861bf4036 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81342) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:59:52.727767 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-eb06374c-3784-4edb-a72f-2e1d833cc8eb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:52.728372 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-eb06374c-3784-4edb-a72f-2e1d833cc8eb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:52.730351 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1043/2077] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Sep 1 14:59:52 2022] DELETE /identity/v3/projects/baf1a0cc82104a8ea760e2e861bf4036 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:59:52.732149 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:59:52.734538 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb06374c-3784-4edb-a72f-2e1d833cc8eb admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:59:52.736232 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-eb06374c-3784-4edb-a72f-2e1d833cc8eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:59:52.786693 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1656417-2aa3-4533-8b8e-ae35b358b773 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3b81739e34e4e648f88b6d73fcb5e46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:59:52.787484 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1656417-2aa3-4533-8b8e-ae35b358b773 service cinder] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:52.787906 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1656417-2aa3-4533-8b8e-ae35b358b773 service cinder] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:52.788403 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1656417-2aa3-4533-8b8e-ae35b358b773 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:52.790694 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-e1656417-2aa3-4533-8b8e-ae35b358b773 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:59:52.825355 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb06374c-3784-4edb-a72f-2e1d833cc8eb admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:59:52.832762 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04410d61-1fdd-472c-b04c-4e45fc9805f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:59:52.833921 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04410d61-1fdd-472c-b04c-4e45fc9805f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:52.834436 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04410d61-1fdd-472c-b04c-4e45fc9805f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:52.834987 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04410d61-1fdd-472c-b04c-4e45fc9805f4 admin admin] PATH_INFO: `/v3/users/2e23d8d81e9841ada564a1481680a97e` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:52.836644 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04410d61-1fdd-472c-b04c-4e45fc9805f4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2e23d8d81e9841ada564a1481680a97e)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:59:52.838890 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04410d61-1fdd-472c-b04c-4e45fc9805f4 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:59:52.887456 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1656417-2aa3-4533-8b8e-ae35b358b773 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:59:52.888866 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1656417-2aa3-4533-8b8e-ae35b358b773 service cinder] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:59:52.898800 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1035/2078] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 14:59:52 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 265 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 14:59:52.902108 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-04410d61-1fdd-472c-b04c-4e45fc9805f4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2e23d8d81e9841ada564a1481680a97e'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:52.915433 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-04410d61-1fdd-472c-b04c-4e45fc9805f4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2e23d8d81e9841ada564a1481680a97e'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:52.916873 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-04410d61-1fdd-472c-b04c-4e45fc9805f4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2e23d8d81e9841ada564a1481680a97e'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:52.925100 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:59:52.934616 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-04410d61-1fdd-472c-b04c-4e45fc9805f4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2e23d8d81e9841ada564a1481680a97e'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:52.935388 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-04410d61-1fdd-472c-b04c-4e45fc9805f4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2e23d8d81e9841ada564a1481680a97e'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:52.949620 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1044/2079] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Sep 1 14:59:52 2022] DELETE /identity/v3/users/2e23d8d81e9841ada564a1481680a97e => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:59:53.263339 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1656417-2aa3-4533-8b8e-ae35b358b773 service cinder] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:59:53.283127 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1656417-2aa3-4533-8b8e-ae35b358b773 service cinder] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:59:53.350873 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f181e41c-3207-4560-8676-aca86f7a0bef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:59:53.351530 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f181e41c-3207-4560-8676-aca86f7a0bef admin admin] REQUEST_METHOD: `DELETE` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:53.351815 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f181e41c-3207-4560-8676-aca86f7a0bef admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:53.352088 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f181e41c-3207-4560-8676-aca86f7a0bef admin admin] PATH_INFO: `/v3/projects/d30ddfb1adee468bba325b4d823778fe` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:53.353213 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f181e41c-3207-4560-8676-aca86f7a0bef admin admin] RBAC: Authorizing `identity:delete_project(project_id=d30ddfb1adee468bba325b4d823778fe)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:59:53.355169 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f181e41c-3207-4560-8676-aca86f7a0bef admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:59:53.401458 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-f181e41c-3207-4560-8676-aca86f7a0bef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd30ddfb1adee468bba325b4d823778fe', 'request_id': 'req-f181e41c-3207-4560-8676-aca86f7a0bef'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:53.407547 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-f181e41c-3207-4560-8676-aca86f7a0bef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd30ddfb1adee468bba325b4d823778fe', 'request_id': 'req-f181e41c-3207-4560-8676-aca86f7a0bef'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:53.414225 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-f181e41c-3207-4560-8676-aca86f7a0bef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd30ddfb1adee468bba325b4d823778fe', 'request_id': 'req-f181e41c-3207-4560-8676-aca86f7a0bef'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:53.425407 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-f181e41c-3207-4560-8676-aca86f7a0bef admin admin] The token cache is being invalidate because project d30ddfb1adee468bba325b4d823778fe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81341) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:59:53.425711 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-f181e41c-3207-4560-8676-aca86f7a0bef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:53.426488 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-f181e41c-3207-4560-8676-aca86f7a0bef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:53.428502 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1036/2080] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Sep 1 14:59:53 2022] DELETE /identity/v3/projects/d30ddfb1adee468bba325b4d823778fe => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:59:53.433226 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f23a11-bc07-4ec2-a0d2-1362550c68d7 None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:53.433675 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f23a11-bc07-4ec2-a0d2-1362550c68d7 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:53.433799 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f23a11-bc07-4ec2-a0d2-1362550c68d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:53.455068 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-d5f23a11-bc07-4ec2-a0d2-1362550c68d7 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:59:53.456280 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-d5f23a11-bc07-4ec2-a0d2-1362550c68d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:59:53.517102 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1045/2081] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 14:59:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:59:53.525065 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8252407d-2c4a-44f8-a3e3-cdc0d6739576 None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:53.525520 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8252407d-2c4a-44f8-a3e3-cdc0d6739576 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:53.525610 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8252407d-2c4a-44f8-a3e3-cdc0d6739576 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:53.539687 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:59:53.545900 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-8252407d-2c4a-44f8-a3e3-cdc0d6739576 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:59:53.548412 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-8252407d-2c4a-44f8-a3e3-cdc0d6739576 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:59:53.558824 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1037/2082] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 14:59:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:59:53.563020 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f23a11-bc07-4ec2-a0d2-1362550c68d7 None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:59:53.573418 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f23a11-bc07-4ec2-a0d2-1362550c68d7 None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:59:53.578139 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7d788e-f5f6-4e2f-81a5-cf9aa6cdf1b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:59:53.578786 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7d788e-f5f6-4e2f-81a5-cf9aa6cdf1b8 admin admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:53.579072 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7d788e-f5f6-4e2f-81a5-cf9aa6cdf1b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:53.579344 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7d788e-f5f6-4e2f-81a5-cf9aa6cdf1b8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:53.579997 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7d788e-f5f6-4e2f-81a5-cf9aa6cdf1b8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:59:53.580300 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7d788e-f5f6-4e2f-81a5-cf9aa6cdf1b8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:59:53.581927 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7d788e-f5f6-4e2f-81a5-cf9aa6cdf1b8 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:59:53.593971 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1046/2083] 10.222.0.33 () {58 vars in 1153 bytes} [Thu Sep 1 14:59:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:59:53.598393 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8252407d-2c4a-44f8-a3e3-cdc0d6739576 None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:59:53.607919 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8252407d-2c4a-44f8-a3e3-cdc0d6739576 None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:59:53.613197 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffb5d93-abf3-4514-9f83-97cdcfa31a4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:59:53.613790 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffb5d93-abf3-4514-9f83-97cdcfa31a4a admin admin] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:53.614069 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffb5d93-abf3-4514-9f83-97cdcfa31a4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:53.614371 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffb5d93-abf3-4514-9f83-97cdcfa31a4a admin admin] PATH_INFO: `/v3/projects` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:53.615014 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffb5d93-abf3-4514-9f83-97cdcfa31a4a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:59:53.616939 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffb5d93-abf3-4514-9f83-97cdcfa31a4a admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:59:53.647676 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1038/2084] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Sep 1 14:59:53 2022] POST /identity/v3/projects => generated 371 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 14:59:53.651850 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7d788e-f5f6-4e2f-81a5-cf9aa6cdf1b8 admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:59:53.664638 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7d788e-f5f6-4e2f-81a5-cf9aa6cdf1b8 admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:59:53.721654 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e154f4-5ffb-42ef-abd8-33b6e32a0e2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:59:53.722294 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e154f4-5ffb-42ef-abd8-33b6e32a0e2e admin admin] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:53.722573 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e154f4-5ffb-42ef-abd8-33b6e32a0e2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:53.722841 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e154f4-5ffb-42ef-abd8-33b6e32a0e2e admin admin] PATH_INFO: `/v3/users` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:53.723487 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e154f4-5ffb-42ef-abd8-33b6e32a0e2e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:59:53.725429 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e154f4-5ffb-42ef-abd8-33b6e32a0e2e admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:59:53.746743 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1047/2085] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Sep 1 14:59:53 2022] POST /identity/v3/users => generated 286 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:59:53.750810 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffb5d93-abf3-4514-9f83-97cdcfa31a4a admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:59:53.761083 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffb5d93-abf3-4514-9f83-97cdcfa31a4a admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:59:53.766427 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e2523b-9345-45c6-a7a0-a266b58b7858 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:59:53.767042 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e2523b-9345-45c6-a7a0-a266b58b7858 admin admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:53.767322 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e2523b-9345-45c6-a7a0-a266b58b7858 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:53.767597 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e2523b-9345-45c6-a7a0-a266b58b7858 admin admin] PATH_INFO: `/v3/roles` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:53.768241 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e2523b-9345-45c6-a7a0-a266b58b7858 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:59:53.769858 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e2523b-9345-45c6-a7a0-a266b58b7858 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:59:53.780124 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1039/2086] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 1 14:59:53 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:59:53.784712 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e154f4-5ffb-42ef-abd8-33b6e32a0e2e admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:59:53.794494 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e154f4-5ffb-42ef-abd8-33b6e32a0e2e admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:59:53.799671 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0828b4-a18d-448b-aaa6-1f29f8a80b88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:59:53.800279 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0828b4-a18d-448b-aaa6-1f29f8a80b88 admin admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:53.800554 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0828b4-a18d-448b-aaa6-1f29f8a80b88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:53.800815 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0828b4-a18d-448b-aaa6-1f29f8a80b88 admin admin] PATH_INFO: `/v3/roles` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:53.801443 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0828b4-a18d-448b-aaa6-1f29f8a80b88 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:59:53.803323 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0828b4-a18d-448b-aaa6-1f29f8a80b88 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:59:53.813764 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1048/2087] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 1 14:59:53 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:59:53.818162 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e2523b-9345-45c6-a7a0-a266b58b7858 admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:59:53.828231 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e2523b-9345-45c6-a7a0-a266b58b7858 admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:59:53.832893 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401c59b6-bcfa-4c91-b8ba-532c431ca9bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:59:53.833432 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401c59b6-bcfa-4c91-b8ba-532c431ca9bd admin admin] REQUEST_METHOD: `PUT` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:53.833733 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401c59b6-bcfa-4c91-b8ba-532c431ca9bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:53.834085 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401c59b6-bcfa-4c91-b8ba-532c431ca9bd admin admin] PATH_INFO: `/v3/projects/54034c9cb0874e87af743396d7ce9061/users/6c2c96fc8b9f419ba8e6b473037d34f9/roles/5d1feef0e13d4874928bd86258c4e3b2` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:53.848866 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401c59b6-bcfa-4c91-b8ba-532c431ca9bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=54034c9cb0874e87af743396d7ce9061, user_id=6c2c96fc8b9f419ba8e6b473037d34f9, role_id=5d1feef0e13d4874928bd86258c4e3b2)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:59:53.851567 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401c59b6-bcfa-4c91-b8ba-532c431ca9bd admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:59:53.864985 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1040/2088] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 1 14:59:53 2022] PUT /identity/v3/projects/54034c9cb0874e87af743396d7ce9061/users/6c2c96fc8b9f419ba8e6b473037d34f9/roles/5d1feef0e13d4874928bd86258c4e3b2 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:59:53.876272 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062ffd88-d5d2-4822-be61-68d32f6f5d98 None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:53.876555 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062ffd88-d5d2-4822-be61-68d32f6f5d98 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:53.876958 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062ffd88-d5d2-4822-be61-68d32f6f5d98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:53.910629 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-062ffd88-d5d2-4822-be61-68d32f6f5d98 None None] MFA Rules not processed for user `6c2c96fc8b9f419ba8e6b473037d34f9`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:59:53.911873 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-062ffd88-d5d2-4822-be61-68d32f6f5d98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:59:53.948667 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:59:53.989363 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1049/2089] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 14:59:53 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:59:53.997810 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401c59b6-bcfa-4c91-b8ba-532c431ca9bd admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:59:54.001095 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-401c59b6-bcfa-4c91-b8ba-532c431ca9bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:59:54.022758 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062ffd88-d5d2-4822-be61-68d32f6f5d98 None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:59:54.024210 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-062ffd88-d5d2-4822-be61-68d32f6f5d98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:59:54.068216 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401c59b6-bcfa-4c91-b8ba-532c431ca9bd admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:59:54.071855 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29b5b98-3ea3-4297-8b28-573b75ada47f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3b81739e34e4e648f88b6d73fcb5e46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:59:54.072490 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29b5b98-3ea3-4297-8b28-573b75ada47f service cinder] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:54.072769 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29b5b98-3ea3-4297-8b28-573b75ada47f service cinder] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:54.073049 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29b5b98-3ea3-4297-8b28-573b75ada47f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:54.085248 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29b5b98-3ea3-4297-8b28-573b75ada47f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:59:54.086741 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29b5b98-3ea3-4297-8b28-573b75ada47f service cinder] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:59:54.096565 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1041/2090] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 14:59:53 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:59:54.110063 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062ffd88-d5d2-4822-be61-68d32f6f5d98 None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:59:54.115226 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e9c5f7-a7cf-4ad2-98aa-28b85b3c7732 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:59:54.115865 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e9c5f7-a7cf-4ad2-98aa-28b85b3c7732 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:54.116903 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e9c5f7-a7cf-4ad2-98aa-28b85b3c7732 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:54.117171 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e9c5f7-a7cf-4ad2-98aa-28b85b3c7732 admin admin] PATH_INFO: `/v3/users/aae2f6b4ac014595a04b1e136ccf13b0` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:54.118610 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e9c5f7-a7cf-4ad2-98aa-28b85b3c7732 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aae2f6b4ac014595a04b1e136ccf13b0)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:59:54.119468 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29b5b98-3ea3-4297-8b28-573b75ada47f service cinder] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:59:54.120564 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e9c5f7-a7cf-4ad2-98aa-28b85b3c7732 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:59:54.136781 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29b5b98-3ea3-4297-8b28-573b75ada47f service cinder] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:59:54.141919 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb1d15f-f5a5-4989-aabd-505bf2b4516c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:59:54.143113 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb1d15f-f5a5-4989-aabd-505bf2b4516c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:54.143591 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb1d15f-f5a5-4989-aabd-505bf2b4516c admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:54.143863 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb1d15f-f5a5-4989-aabd-505bf2b4516c admin admin] PATH_INFO: `/v3/users/6c2c96fc8b9f419ba8e6b473037d34f9` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:54.145307 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb1d15f-f5a5-4989-aabd-505bf2b4516c admin admin] RBAC: Authorizing `identity:delete_user(user_id=6c2c96fc8b9f419ba8e6b473037d34f9)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:59:54.147505 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb1d15f-f5a5-4989-aabd-505bf2b4516c admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:59:54.171775 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-a2e9c5f7-a7cf-4ad2-98aa-28b85b3c7732 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aae2f6b4ac014595a04b1e136ccf13b0'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:54.182629 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-a2e9c5f7-a7cf-4ad2-98aa-28b85b3c7732 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aae2f6b4ac014595a04b1e136ccf13b0'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:54.183028 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-a2e9c5f7-a7cf-4ad2-98aa-28b85b3c7732 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aae2f6b4ac014595a04b1e136ccf13b0'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:54.197213 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-a2e9c5f7-a7cf-4ad2-98aa-28b85b3c7732 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aae2f6b4ac014595a04b1e136ccf13b0'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:54.198044 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-a2e9c5f7-a7cf-4ad2-98aa-28b85b3c7732 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aae2f6b4ac014595a04b1e136ccf13b0'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:54.206066 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-7cb1d15f-f5a5-4989-aabd-505bf2b4516c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6c2c96fc8b9f419ba8e6b473037d34f9'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:54.211247 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1050/2091] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Sep 1 14:59:54 2022] DELETE /identity/v3/users/aae2f6b4ac014595a04b1e136ccf13b0 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 14:59:54.215675 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11d58f3-9cc9-43c7-8888-57ca550ce9d4 None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:54.216068 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11d58f3-9cc9-43c7-8888-57ca550ce9d4 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:54.216405 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11d58f3-9cc9-43c7-8888-57ca550ce9d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:54.216501 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-7cb1d15f-f5a5-4989-aabd-505bf2b4516c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6c2c96fc8b9f419ba8e6b473037d34f9'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:54.217161 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-7cb1d15f-f5a5-4989-aabd-505bf2b4516c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6c2c96fc8b9f419ba8e6b473037d34f9'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:54.231921 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-7cb1d15f-f5a5-4989-aabd-505bf2b4516c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6c2c96fc8b9f419ba8e6b473037d34f9'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:54.232785 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-7cb1d15f-f5a5-4989-aabd-505bf2b4516c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6c2c96fc8b9f419ba8e6b473037d34f9'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:54.237020 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-a11d58f3-9cc9-43c7-8888-57ca550ce9d4 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:59:54.238260 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-a11d58f3-9cc9-43c7-8888-57ca550ce9d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:59:54.245119 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1042/2092] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Sep 1 14:59:54 2022] DELETE /identity/v3/users/6c2c96fc8b9f419ba8e6b473037d34f9 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 14:59:54.249395 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdde63c-03b6-4e12-bbd8-c48522356ce0 None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:54.249707 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdde63c-03b6-4e12-bbd8-c48522356ce0 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:54.249983 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdde63c-03b6-4e12-bbd8-c48522356ce0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:54.269929 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-3fdde63c-03b6-4e12-bbd8-c48522356ce0 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:59:54.271211 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-3fdde63c-03b6-4e12-bbd8-c48522356ce0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:59:54.284900 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:59:54.288520 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:59:54.306874 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1051/2093] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 14:59:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 14:59:54.320591 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11d58f3-9cc9-43c7-8888-57ca550ce9d4 None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:59:54.322096 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-a11d58f3-9cc9-43c7-8888-57ca550ce9d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:59:54.347475 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1043/2094] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 14:59:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 14:59:54.361195 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdde63c-03b6-4e12-bbd8-c48522356ce0 None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:59:54.362684 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-3fdde63c-03b6-4e12-bbd8-c48522356ce0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:59:54.400918 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11d58f3-9cc9-43c7-8888-57ca550ce9d4 None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:59:54.449138 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdde63c-03b6-4e12-bbd8-c48522356ce0 None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:59:54.453627 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268f1c3d-0c7c-443a-bebe-4ce229ed6d24 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2790149639aa418bbc4e4982e93eda44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:59:54.454258 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268f1c3d-0c7c-443a-bebe-4ce229ed6d24 service neutron] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:54.454539 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268f1c3d-0c7c-443a-bebe-4ce229ed6d24 service neutron] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:54.454804 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268f1c3d-0c7c-443a-bebe-4ce229ed6d24 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:54.465685 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268f1c3d-0c7c-443a-bebe-4ce229ed6d24 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:59:54.467387 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268f1c3d-0c7c-443a-bebe-4ce229ed6d24 service neutron] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:59:54.469493 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d642d73-a8aa-4785-bbc2-eb1cde29f75c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2790149639aa418bbc4e4982e93eda44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:59:54.470169 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d642d73-a8aa-4785-bbc2-eb1cde29f75c service neutron] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:54.470573 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d642d73-a8aa-4785-bbc2-eb1cde29f75c service neutron] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:54.470917 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d642d73-a8aa-4785-bbc2-eb1cde29f75c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:54.477425 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1044/2095] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 14:59:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 14:59:54.481739 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d642d73-a8aa-4785-bbc2-eb1cde29f75c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:59:54.483918 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d642d73-a8aa-4785-bbc2-eb1cde29f75c service neutron] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:59:54.496059 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1052/2096] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 14:59:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 14:59:54.867990 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268f1c3d-0c7c-443a-bebe-4ce229ed6d24 service neutron] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:59:54.869699 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-268f1c3d-0c7c-443a-bebe-4ce229ed6d24 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:59:54.884101 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d642d73-a8aa-4785-bbc2-eb1cde29f75c service neutron] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:59:54.885738 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-0d642d73-a8aa-4785-bbc2-eb1cde29f75c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:59:54.900343 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d642d73-a8aa-4785-bbc2-eb1cde29f75c service neutron] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:59:54.904902 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c8fa57-7253-4a7d-aa72-a67e6e6f2af6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:59:54.905521 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c8fa57-7253-4a7d-aa72-a67e6e6f2af6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:54.905798 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c8fa57-7253-4a7d-aa72-a67e6e6f2af6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:54.906067 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c8fa57-7253-4a7d-aa72-a67e6e6f2af6 admin admin] PATH_INFO: `/v3/projects/96a9dae3808140258407c30997c62b73` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:54.907219 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c8fa57-7253-4a7d-aa72-a67e6e6f2af6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=96a9dae3808140258407c30997c62b73)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:59:54.909000 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268f1c3d-0c7c-443a-bebe-4ce229ed6d24 service neutron] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:59:54.909471 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c8fa57-7253-4a7d-aa72-a67e6e6f2af6 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:59:54.922344 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2105a65f-26d6-49de-90a3-b8b0c9be4f66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:59:54.923050 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2105a65f-26d6-49de-90a3-b8b0c9be4f66 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:54.923249 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2105a65f-26d6-49de-90a3-b8b0c9be4f66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:54.923519 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2105a65f-26d6-49de-90a3-b8b0c9be4f66 admin admin] PATH_INFO: `/v3/projects/54034c9cb0874e87af743396d7ce9061` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:54.925193 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2105a65f-26d6-49de-90a3-b8b0c9be4f66 admin admin] RBAC: Authorizing `identity:delete_project(project_id=54034c9cb0874e87af743396d7ce9061)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:59:54.927129 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2105a65f-26d6-49de-90a3-b8b0c9be4f66 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:59:54.964678 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-05c8fa57-7253-4a7d-aa72-a67e6e6f2af6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '96a9dae3808140258407c30997c62b73', 'request_id': 'req-05c8fa57-7253-4a7d-aa72-a67e6e6f2af6'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:54.971289 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-05c8fa57-7253-4a7d-aa72-a67e6e6f2af6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '96a9dae3808140258407c30997c62b73', 'request_id': 'req-05c8fa57-7253-4a7d-aa72-a67e6e6f2af6'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:54.978388 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-2105a65f-26d6-49de-90a3-b8b0c9be4f66 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '54034c9cb0874e87af743396d7ce9061', 'request_id': 'req-2105a65f-26d6-49de-90a3-b8b0c9be4f66'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:54.979693 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-05c8fa57-7253-4a7d-aa72-a67e6e6f2af6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '96a9dae3808140258407c30997c62b73', 'request_id': 'req-05c8fa57-7253-4a7d-aa72-a67e6e6f2af6'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:54.985547 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-2105a65f-26d6-49de-90a3-b8b0c9be4f66 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '54034c9cb0874e87af743396d7ce9061', 'request_id': 'req-2105a65f-26d6-49de-90a3-b8b0c9be4f66'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:54.992395 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-05c8fa57-7253-4a7d-aa72-a67e6e6f2af6 admin admin] The token cache is being invalidate because project 96a9dae3808140258407c30997c62b73 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81342) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:59:54.992682 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-05c8fa57-7253-4a7d-aa72-a67e6e6f2af6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:54.993346 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-05c8fa57-7253-4a7d-aa72-a67e6e6f2af6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:54.994143 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-2105a65f-26d6-49de-90a3-b8b0c9be4f66 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '54034c9cb0874e87af743396d7ce9061', 'request_id': 'req-2105a65f-26d6-49de-90a3-b8b0c9be4f66'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:54.994952 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1053/2097] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Sep 1 14:59:54 2022] DELETE /identity/v3/projects/96a9dae3808140258407c30997c62b73 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 14:59:54.999797 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d5b1ce-c0fc-4b0e-bdfe-0ad1659be1de None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:55.000082 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d5b1ce-c0fc-4b0e-bdfe-0ad1659be1de None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:55.000357 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d5b1ce-c0fc-4b0e-bdfe-0ad1659be1de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:55.008368 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-2105a65f-26d6-49de-90a3-b8b0c9be4f66 admin admin] The token cache is being invalidate because project 54034c9cb0874e87af743396d7ce9061 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81341) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:59:55.008662 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-2105a65f-26d6-49de-90a3-b8b0c9be4f66 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:55.009318 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-2105a65f-26d6-49de-90a3-b8b0c9be4f66 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:55.011048 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1045/2098] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Sep 1 14:59:54 2022] DELETE /identity/v3/projects/54034c9cb0874e87af743396d7ce9061 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:59:55.015493 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb09195-e7cd-4b96-a20e-98cf20509b6e None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:55.015783 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb09195-e7cd-4b96-a20e-98cf20509b6e None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:55.016065 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb09195-e7cd-4b96-a20e-98cf20509b6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:55.021443 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-b4d5b1ce-c0fc-4b0e-bdfe-0ad1659be1de None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:59:55.022877 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-b4d5b1ce-c0fc-4b0e-bdfe-0ad1659be1de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:59:55.038589 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-5cb09195-e7cd-4b96-a20e-98cf20509b6e None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:59:55.039885 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-5cb09195-e7cd-4b96-a20e-98cf20509b6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:59:55.055852 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:59:55.084864 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:59:55.088425 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1054/2099] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 14:59:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 14:59:55.098305 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625c19e6-0b7b-4d09-82ca-95f690556a03 None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:55.098708 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625c19e6-0b7b-4d09-82ca-95f690556a03 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:55.099011 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625c19e6-0b7b-4d09-82ca-95f690556a03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:55.106735 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1046/2100] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 14:59:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 14:59:55.114769 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6259e2-6996-43a8-9167-a522df4f95d3 None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:55.115049 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6259e2-6996-43a8-9167-a522df4f95d3 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:55.115318 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6259e2-6996-43a8-9167-a522df4f95d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:55.123615 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-625c19e6-0b7b-4d09-82ca-95f690556a03 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:59:55.124819 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-625c19e6-0b7b-4d09-82ca-95f690556a03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:59:55.136037 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-4f6259e2-6996-43a8-9167-a522df4f95d3 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:59:55.136395 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-4f6259e2-6996-43a8-9167-a522df4f95d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:59:55.137088 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1055/2101] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 14:59:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 14:59:55.138015 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625c19e6-0b7b-4d09-82ca-95f690556a03 None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:59:55.145024 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1047/2102] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 14:59:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 14:59:55.148218 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625c19e6-0b7b-4d09-82ca-95f690556a03 None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:59:55.148991 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6259e2-6996-43a8-9167-a522df4f95d3 None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:59:55.153020 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09323c8b-5a2d-4c7f-9766-e71c67a1d035 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:59:55.153636 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09323c8b-5a2d-4c7f-9766-e71c67a1d035 admin admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:55.154876 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09323c8b-5a2d-4c7f-9766-e71c67a1d035 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:55.155165 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09323c8b-5a2d-4c7f-9766-e71c67a1d035 admin admin] PATH_INFO: `/v3/domains` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:55.155827 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09323c8b-5a2d-4c7f-9766-e71c67a1d035 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:59:55.156130 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09323c8b-5a2d-4c7f-9766-e71c67a1d035 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:59:55.157751 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09323c8b-5a2d-4c7f-9766-e71c67a1d035 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:59:55.158778 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6259e2-6996-43a8-9167-a522df4f95d3 None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:59:55.164856 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0844a9f-3b5b-4c8a-af30-b83704b20b95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:59:55.165829 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0844a9f-3b5b-4c8a-af30-b83704b20b95 admin admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:55.165829 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0844a9f-3b5b-4c8a-af30-b83704b20b95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:55.166034 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0844a9f-3b5b-4c8a-af30-b83704b20b95 admin admin] PATH_INFO: `/v3/domains` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:55.166633 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0844a9f-3b5b-4c8a-af30-b83704b20b95 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81341) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:59:55.166930 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0844a9f-3b5b-4c8a-af30-b83704b20b95 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:59:55.168526 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0844a9f-3b5b-4c8a-af30-b83704b20b95 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:59:55.171558 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1056/2103] 10.222.0.33 () {58 vars in 1153 bytes} [Thu Sep 1 14:59:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:59:55.176607 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09323c8b-5a2d-4c7f-9766-e71c67a1d035 admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:59:55.186424 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1048/2104] 10.222.0.33 () {58 vars in 1153 bytes} [Thu Sep 1 14:59:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 14:59:55.187797 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0844a9f-3b5b-4c8a-af30-b83704b20b95 admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:59:55.188208 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09323c8b-5a2d-4c7f-9766-e71c67a1d035 admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:59:55.193114 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840e8729-e605-494f-bf35-36d4e0b7e6c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:59:55.193706 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840e8729-e605-494f-bf35-36d4e0b7e6c5 admin admin] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:55.194571 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840e8729-e605-494f-bf35-36d4e0b7e6c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:55.194571 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840e8729-e605-494f-bf35-36d4e0b7e6c5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:55.194907 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840e8729-e605-494f-bf35-36d4e0b7e6c5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:59:55.197200 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840e8729-e605-494f-bf35-36d4e0b7e6c5 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:59:55.205831 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0844a9f-3b5b-4c8a-af30-b83704b20b95 admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:59:55.210990 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22412557-8f4c-46fa-87db-549e7cc921fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:59:55.212015 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22412557-8f4c-46fa-87db-549e7cc921fb admin admin] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:55.212412 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22412557-8f4c-46fa-87db-549e7cc921fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:55.212577 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22412557-8f4c-46fa-87db-549e7cc921fb admin admin] PATH_INFO: `/v3/projects` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:55.213213 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22412557-8f4c-46fa-87db-549e7cc921fb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:59:55.215259 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22412557-8f4c-46fa-87db-549e7cc921fb admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:59:55.233186 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1057/2105] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Sep 1 14:59:55 2022] POST /identity/v3/projects => generated 405 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 14:59:55.237965 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840e8729-e605-494f-bf35-36d4e0b7e6c5 admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:59:55.251485 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1049/2106] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Sep 1 14:59:55 2022] POST /identity/v3/projects => generated 369 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 14:59:55.255254 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22412557-8f4c-46fa-87db-549e7cc921fb admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:59:55.315401 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840e8729-e605-494f-bf35-36d4e0b7e6c5 admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:59:55.333998 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22412557-8f4c-46fa-87db-549e7cc921fb admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:59:55.338731 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84112e79-48b4-44c2-bbf4-bbd09c32403c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:59:55.339333 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84112e79-48b4-44c2-bbf4-bbd09c32403c admin admin] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:55.339609 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84112e79-48b4-44c2-bbf4-bbd09c32403c admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:55.339876 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84112e79-48b4-44c2-bbf4-bbd09c32403c admin admin] PATH_INFO: `/v3/users` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:55.340519 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84112e79-48b4-44c2-bbf4-bbd09c32403c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:59:55.342460 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84112e79-48b4-44c2-bbf4-bbd09c32403c admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:59:55.368260 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1050/2107] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Sep 1 14:59:55 2022] POST /identity/v3/users => generated 285 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 14:59:55.369221 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84112e79-48b4-44c2-bbf4-bbd09c32403c admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:59:55.375610 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83155cf-870f-437b-9b9a-36054311f2ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:59:55.376216 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83155cf-870f-437b-9b9a-36054311f2ee admin admin] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:55.376492 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83155cf-870f-437b-9b9a-36054311f2ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:55.376798 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83155cf-870f-437b-9b9a-36054311f2ee admin admin] PATH_INFO: `/v3/users` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:55.377424 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83155cf-870f-437b-9b9a-36054311f2ee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:59:55.379490 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84112e79-48b4-44c2-bbf4-bbd09c32403c admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:59:55.379602 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83155cf-870f-437b-9b9a-36054311f2ee admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:59:55.384150 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47687f4-1c1a-4e94-8128-23998311ada0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:59:55.384734 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47687f4-1c1a-4e94-8128-23998311ada0 admin admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:55.385011 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47687f4-1c1a-4e94-8128-23998311ada0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:55.385281 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47687f4-1c1a-4e94-8128-23998311ada0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:55.385919 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47687f4-1c1a-4e94-8128-23998311ada0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:59:55.387546 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47687f4-1c1a-4e94-8128-23998311ada0 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:59:55.398071 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1051/2108] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 1 14:59:55 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:59:55.404660 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1058/2109] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Sep 1 14:59:55 2022] POST /identity/v3/users => generated 303 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 14:59:55.404794 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47687f4-1c1a-4e94-8128-23998311ada0 admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:59:55.405601 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83155cf-870f-437b-9b9a-36054311f2ee admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:59:55.411969 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47687f4-1c1a-4e94-8128-23998311ada0 admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:59:55.415959 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83155cf-870f-437b-9b9a-36054311f2ee admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:59:55.416761 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e4c2ca-c9e5-446c-8242-d6b47820ad4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:59:55.417354 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e4c2ca-c9e5-446c-8242-d6b47820ad4f admin admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:55.417636 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e4c2ca-c9e5-446c-8242-d6b47820ad4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:55.417905 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e4c2ca-c9e5-446c-8242-d6b47820ad4f admin admin] PATH_INFO: `/v3/roles` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:55.418573 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e4c2ca-c9e5-446c-8242-d6b47820ad4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:59:55.420317 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e4c2ca-c9e5-446c-8242-d6b47820ad4f admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:59:55.425843 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83421c05-2381-4f61-9c8e-4fd454a8ce11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:59:55.426501 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83421c05-2381-4f61-9c8e-4fd454a8ce11 admin admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:55.426824 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83421c05-2381-4f61-9c8e-4fd454a8ce11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:55.427096 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83421c05-2381-4f61-9c8e-4fd454a8ce11 admin admin] PATH_INFO: `/v3/roles` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:55.428145 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83421c05-2381-4f61-9c8e-4fd454a8ce11 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:59:55.430733 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83421c05-2381-4f61-9c8e-4fd454a8ce11 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:59:55.434494 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1052/2110] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 1 14:59:55 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 14:59:55.438594 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e4c2ca-c9e5-446c-8242-d6b47820ad4f admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:59:55.446944 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1059/2111] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 1 14:59:55 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 14:59:55.448051 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83421c05-2381-4f61-9c8e-4fd454a8ce11 admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:59:55.451451 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e4c2ca-c9e5-446c-8242-d6b47820ad4f admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:59:55.456258 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0dd792-acd6-4b60-8689-420f97c90e57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:59:55.456982 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0dd792-acd6-4b60-8689-420f97c90e57 admin admin] REQUEST_METHOD: `PUT` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:55.457333 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0dd792-acd6-4b60-8689-420f97c90e57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:55.457674 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0dd792-acd6-4b60-8689-420f97c90e57 admin admin] PATH_INFO: `/v3/projects/fba2b42d3ca24ba7901bc9e3cc2ed8b5/users/7787af243c63452dadb2b9a957c43fda/roles/5d1feef0e13d4874928bd86258c4e3b2` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:55.458712 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83421c05-2381-4f61-9c8e-4fd454a8ce11 admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:59:55.464245 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c49f19-c856-4495-a271-0f6e38a7eb7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:59:55.464912 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c49f19-c856-4495-a271-0f6e38a7eb7e admin admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:55.465358 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c49f19-c856-4495-a271-0f6e38a7eb7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:55.477948 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c49f19-c856-4495-a271-0f6e38a7eb7e admin admin] PATH_INFO: `/v3/roles` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:55.477948 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c49f19-c856-4495-a271-0f6e38a7eb7e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:59:55.477948 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c49f19-c856-4495-a271-0f6e38a7eb7e admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:59:55.479795 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0dd792-acd6-4b60-8689-420f97c90e57 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fba2b42d3ca24ba7901bc9e3cc2ed8b5, user_id=7787af243c63452dadb2b9a957c43fda, role_id=5d1feef0e13d4874928bd86258c4e3b2)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:59:55.481447 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1060/2112] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 1 14:59:55 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:59:55.482714 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0dd792-acd6-4b60-8689-420f97c90e57 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:59:55.485533 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c49f19-c856-4495-a271-0f6e38a7eb7e admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:59:55.496590 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c49f19-c856-4495-a271-0f6e38a7eb7e admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:59:55.497549 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1053/2113] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 1 14:59:55 2022] PUT /identity/v3/projects/fba2b42d3ca24ba7901bc9e3cc2ed8b5/users/7787af243c63452dadb2b9a957c43fda/roles/5d1feef0e13d4874928bd86258c4e3b2 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:59:55.503309 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:59:55.508156 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf79d5c5-6f58-4519-b09c-af5831f63fcd None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:55.508438 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf79d5c5-6f58-4519-b09c-af5831f63fcd None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:55.508736 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf79d5c5-6f58-4519-b09c-af5831f63fcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:55.540778 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-bf79d5c5-6f58-4519-b09c-af5831f63fcd None None] MFA Rules not processed for user `7787af243c63452dadb2b9a957c43fda`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:59:55.542108 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-bf79d5c5-6f58-4519-b09c-af5831f63fcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:59:55.567803 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1500955-b21c-43fb-9675-19d3e66c7572 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:59:55.567803 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1500955-b21c-43fb-9675-19d3e66c7572 admin admin] REQUEST_METHOD: `PUT` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:55.567803 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1500955-b21c-43fb-9675-19d3e66c7572 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:55.568038 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1500955-b21c-43fb-9675-19d3e66c7572 admin admin] PATH_INFO: `/v3/projects/65676a714773423c8bf8a05c985c70ff/users/a5291d0613f8401192716fe7f468c72b/roles/5d1feef0e13d4874928bd86258c4e3b2` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:55.578152 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:59:55.584814 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1500955-b21c-43fb-9675-19d3e66c7572 admin admin] RBAC: Authorizing `identity:create_grant(project_id=65676a714773423c8bf8a05c985c70ff, user_id=a5291d0613f8401192716fe7f468c72b, role_id=5d1feef0e13d4874928bd86258c4e3b2)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:59:55.587925 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1500955-b21c-43fb-9675-19d3e66c7572 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:59:55.601876 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1061/2114] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 1 14:59:55 2022] PUT /identity/v3/projects/65676a714773423c8bf8a05c985c70ff/users/a5291d0613f8401192716fe7f468c72b/roles/5d1feef0e13d4874928bd86258c4e3b2 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 14:59:55.607111 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58764cc7-ebae-4784-874d-909652024be7 None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:55.607392 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58764cc7-ebae-4784-874d-909652024be7 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:55.607687 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58764cc7-ebae-4784-874d-909652024be7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:55.612810 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1054/2115] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 14:59:55 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:59:55.624728 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf79d5c5-6f58-4519-b09c-af5831f63fcd None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:59:55.626137 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-bf79d5c5-6f58-4519-b09c-af5831f63fcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:59:55.627349 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-58764cc7-ebae-4784-874d-909652024be7 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:59:55.628670 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-58764cc7-ebae-4784-874d-909652024be7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:59:55.690605 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf79d5c5-6f58-4519-b09c-af5831f63fcd None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:59:55.696698 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1062/2116] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 14:59:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 14:59:55.707164 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58764cc7-ebae-4784-874d-909652024be7 None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:59:55.708549 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-58764cc7-ebae-4784-874d-909652024be7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:59:55.742964 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e51d20b-5cc3-4c20-8c5d-7410f6871197 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3069f72ce98049b99c9d98229a62bb51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:59:55.743845 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e51d20b-5cc3-4c20-8c5d-7410f6871197 service glance] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:55.744118 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e51d20b-5cc3-4c20-8c5d-7410f6871197 service glance] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:55.744389 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e51d20b-5cc3-4c20-8c5d-7410f6871197 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:55.768319 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58764cc7-ebae-4784-874d-909652024be7 None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:59:55.772027 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258869cb-8ec0-4e8e-b26a-0316ff7dc641 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2790149639aa418bbc4e4982e93eda44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:59:55.772632 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258869cb-8ec0-4e8e-b26a-0316ff7dc641 service neutron] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:55.772910 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258869cb-8ec0-4e8e-b26a-0316ff7dc641 service neutron] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:55.773178 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258869cb-8ec0-4e8e-b26a-0316ff7dc641 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:55.784121 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258869cb-8ec0-4e8e-b26a-0316ff7dc641 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:59:55.785876 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258869cb-8ec0-4e8e-b26a-0316ff7dc641 service neutron] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:59:55.795482 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1063/2117] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 14:59:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 14:59:55.801818 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e51d20b-5cc3-4c20-8c5d-7410f6871197 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:59:55.803560 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e51d20b-5cc3-4c20-8c5d-7410f6871197 service glance] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:59:55.813893 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1055/2118] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 14:59:55 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 14:59:56.491977 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01258a38-d3a8-4d27-9e7b-01f3c827e9e6 None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:56.492477 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01258a38-d3a8-4d27-9e7b-01f3c827e9e6 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:56.492597 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01258a38-d3a8-4d27-9e7b-01f3c827e9e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:56.531376 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-01258a38-d3a8-4d27-9e7b-01f3c827e9e6 None None] MFA Rules not processed for user `a5291d0613f8401192716fe7f468c72b`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:59:56.532705 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-01258a38-d3a8-4d27-9e7b-01f3c827e9e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:59:56.576926 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:59:56.621127 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1064/2119] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 14:59:56 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:59:56.625618 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e51d20b-5cc3-4c20-8c5d-7410f6871197 service glance] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:59:56.636306 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e51d20b-5cc3-4c20-8c5d-7410f6871197 service glance] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:59:56.640988 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195674c3-2f60-4b61-a884-57fbfbd2d1df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:59:56.641583 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195674c3-2f60-4b61-a884-57fbfbd2d1df admin admin] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:56.641864 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195674c3-2f60-4b61-a884-57fbfbd2d1df admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:56.642140 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195674c3-2f60-4b61-a884-57fbfbd2d1df admin admin] PATH_INFO: `/v3/projects` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:56.643321 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195674c3-2f60-4b61-a884-57fbfbd2d1df admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:59:56.645230 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195674c3-2f60-4b61-a884-57fbfbd2d1df admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:59:56.676257 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1056/2120] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Sep 1 14:59:56 2022] POST /identity/v3/projects => generated 405 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 14:59:56.680516 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01258a38-d3a8-4d27-9e7b-01f3c827e9e6 None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:59:56.692445 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01258a38-d3a8-4d27-9e7b-01f3c827e9e6 None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:59:56.748387 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e749ad-0e8a-4951-ad84-2244e1c9d9d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:59:56.748921 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e749ad-0e8a-4951-ad84-2244e1c9d9d5 admin admin] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:56.749200 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e749ad-0e8a-4951-ad84-2244e1c9d9d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:56.749470 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e749ad-0e8a-4951-ad84-2244e1c9d9d5 admin admin] PATH_INFO: `/v3/users` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:56.750107 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e749ad-0e8a-4951-ad84-2244e1c9d9d5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:59:56.752602 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e749ad-0e8a-4951-ad84-2244e1c9d9d5 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:59:56.774286 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1065/2121] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Sep 1 14:59:56 2022] POST /identity/v3/users => generated 309 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:59:56.777577 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195674c3-2f60-4b61-a884-57fbfbd2d1df admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:59:56.787726 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195674c3-2f60-4b61-a884-57fbfbd2d1df admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:59:56.792231 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1558f6c6-47b2-4215-b4fe-571ba951a127 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:59:56.792801 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1558f6c6-47b2-4215-b4fe-571ba951a127 admin admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:56.793080 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1558f6c6-47b2-4215-b4fe-571ba951a127 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:56.793351 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1558f6c6-47b2-4215-b4fe-571ba951a127 admin admin] PATH_INFO: `/v3/roles` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:56.793981 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1558f6c6-47b2-4215-b4fe-571ba951a127 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:59:56.795628 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1558f6c6-47b2-4215-b4fe-571ba951a127 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:59:56.806270 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1057/2122] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 1 14:59:56 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:59:56.810285 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e749ad-0e8a-4951-ad84-2244e1c9d9d5 admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:59:56.820613 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e749ad-0e8a-4951-ad84-2244e1c9d9d5 admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:59:56.825209 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be3551c-33bb-4d4b-887e-676bb957e520 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:59:56.825845 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be3551c-33bb-4d4b-887e-676bb957e520 admin admin] REQUEST_METHOD: `PUT` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:56.826132 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be3551c-33bb-4d4b-887e-676bb957e520 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:56.826444 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be3551c-33bb-4d4b-887e-676bb957e520 admin admin] PATH_INFO: `/v3/projects/5b73abfdf86a45e28ef866b625e15b1c/users/bd1b0d29e0924a129f35dbe5f0ddfcfd/roles/2a2adb769e55476b82e5014897e0a9c0` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:56.842827 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be3551c-33bb-4d4b-887e-676bb957e520 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b73abfdf86a45e28ef866b625e15b1c, user_id=bd1b0d29e0924a129f35dbe5f0ddfcfd, role_id=2a2adb769e55476b82e5014897e0a9c0)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:59:56.845474 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be3551c-33bb-4d4b-887e-676bb957e520 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:59:56.858137 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1066/2123] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 1 14:59:56 2022] PUT /identity/v3/projects/5b73abfdf86a45e28ef866b625e15b1c/users/bd1b0d29e0924a129f35dbe5f0ddfcfd/roles/2a2adb769e55476b82e5014897e0a9c0 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:59:56.862127 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1558f6c6-47b2-4215-b4fe-571ba951a127 admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:59:56.872194 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1558f6c6-47b2-4215-b4fe-571ba951a127 admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:59:56.881367 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:59:56.928435 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8cfb57-4e2d-4153-bf9c-81d742a4bcbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:59:56.929028 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8cfb57-4e2d-4153-bf9c-81d742a4bcbb admin admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:56.929458 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8cfb57-4e2d-4153-bf9c-81d742a4bcbb admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:56.929803 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8cfb57-4e2d-4153-bf9c-81d742a4bcbb admin admin] PATH_INFO: `/v3/roles` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:56.930531 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8cfb57-4e2d-4153-bf9c-81d742a4bcbb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:59:56.932131 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8cfb57-4e2d-4153-bf9c-81d742a4bcbb admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:59:56.942271 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1058/2124] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 1 14:59:56 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:59:56.946432 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be3551c-33bb-4d4b-887e-676bb957e520 admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:59:56.957220 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be3551c-33bb-4d4b-887e-676bb957e520 admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:59:56.961810 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f437131-a02f-4ad8-9ce1-b66494576cc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:59:56.962471 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f437131-a02f-4ad8-9ce1-b66494576cc4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:56.962750 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f437131-a02f-4ad8-9ce1-b66494576cc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:56.963017 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f437131-a02f-4ad8-9ce1-b66494576cc4 admin admin] PATH_INFO: `/v3/projects/5b73abfdf86a45e28ef866b625e15b1c/users/bd1b0d29e0924a129f35dbe5f0ddfcfd/roles/2a2adb769e55476b82e5014897e0a9c0` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:56.964891 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f437131-a02f-4ad8-9ce1-b66494576cc4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b73abfdf86a45e28ef866b625e15b1c, user_id=bd1b0d29e0924a129f35dbe5f0ddfcfd, role_id=2a2adb769e55476b82e5014897e0a9c0)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:59:56.967571 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f437131-a02f-4ad8-9ce1-b66494576cc4 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:59:56.977547 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1067/2125] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 1 14:59:56 2022] PUT /identity/v3/projects/5b73abfdf86a45e28ef866b625e15b1c/users/bd1b0d29e0924a129f35dbe5f0ddfcfd/roles/2a2adb769e55476b82e5014897e0a9c0 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:59:57.616046 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89844799-e353-405b-b61d-6c0df0e6c711 None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:57.616553 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89844799-e353-405b-b61d-6c0df0e6c711 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:57.617040 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89844799-e353-405b-b61d-6c0df0e6c711 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:57.673306 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-89844799-e353-405b-b61d-6c0df0e6c711 None None] MFA Rules not processed for user `bd1b0d29e0924a129f35dbe5f0ddfcfd`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:59:57.675837 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-89844799-e353-405b-b61d-6c0df0e6c711 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:59:57.772663 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1059/2126] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 14:59:57 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:59:58.833896 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb7f4b8-fd7a-49b4-80d0-d822808c2ab8 None None] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:58.834381 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb7f4b8-fd7a-49b4-80d0-d822808c2ab8 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:58.834680 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb7f4b8-fd7a-49b4-80d0-d822808c2ab8 None None] PATH_INFO: `/` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:58.836725 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1068/2127] 10.222.0.33 () {54 vars in 921 bytes} [Thu Sep 1 14:59:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:59:58.859250 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9e6efc-9a11-4773-9381-9d071990be37 None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:58.859697 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9e6efc-9a11-4773-9381-9d071990be37 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:58.859823 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9e6efc-9a11-4773-9381-9d071990be37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:58.882163 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-fc9e6efc-9a11-4773-9381-9d071990be37 None None] MFA Rules not processed for user `7f312c8dcabe48ef9ea453364ade38ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:59:58.883737 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-fc9e6efc-9a11-4773-9381-9d071990be37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:59:58.912260 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb7f4b8-fd7a-49b4-80d0-d822808c2ab8 None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:59:58.984729 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1060/2128] 10.222.0.33 () {58 vars in 1018 bytes} [Thu Sep 1 14:59:58 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:59:58.997139 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9e6efc-9a11-4773-9381-9d071990be37 None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:59:58.998762 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-fc9e6efc-9a11-4773-9381-9d071990be37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:59:59.005596 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb7f4b8-fd7a-49b4-80d0-d822808c2ab8 None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:59:59.010423 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd60845-5578-46f2-a596-8171ab24f705 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:59:59.011062 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd60845-5578-46f2-a596-8171ab24f705 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:59.012093 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd60845-5578-46f2-a596-8171ab24f705 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:59.012093 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd60845-5578-46f2-a596-8171ab24f705 admin admin] PATH_INFO: `/v3/users/a5291d0613f8401192716fe7f468c72b` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:59.013094 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd60845-5578-46f2-a596-8171ab24f705 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a5291d0613f8401192716fe7f468c72b)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:59:59.015171 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd60845-5578-46f2-a596-8171ab24f705 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:59:59.016199 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9e6efc-9a11-4773-9381-9d071990be37 None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:59:59.021746 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc80078-f376-4930-a9be-0752150efaed service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f312c8dcabe48ef9ea453364ade38ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:59:59.022441 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc80078-f376-4930-a9be-0752150efaed service placement] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:59.022794 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc80078-f376-4930-a9be-0752150efaed service placement] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:59.023136 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc80078-f376-4930-a9be-0752150efaed service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:59.032036 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc80078-f376-4930-a9be-0752150efaed service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:59:59.034221 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc80078-f376-4930-a9be-0752150efaed service placement] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:59:59.038623 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=93,keystone:INSERT=5,keystone:DELETE=2 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:59:59.046177 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1061/2129] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 14:59:58 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 14:59:59.066647 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-2dd60845-5578-46f2-a596-8171ab24f705 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a5291d0613f8401192716fe7f468c72b'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:59.074589 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-2dd60845-5578-46f2-a596-8171ab24f705 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a5291d0613f8401192716fe7f468c72b'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:59.075232 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-2dd60845-5578-46f2-a596-8171ab24f705 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a5291d0613f8401192716fe7f468c72b'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:59.086951 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-2dd60845-5578-46f2-a596-8171ab24f705 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a5291d0613f8401192716fe7f468c72b'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:59.087608 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-2dd60845-5578-46f2-a596-8171ab24f705 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a5291d0613f8401192716fe7f468c72b'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:59.100207 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1069/2130] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Sep 1 14:59:58 2022] DELETE /identity/v3/users/a5291d0613f8401192716fe7f468c72b => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 14:59:59.272924 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc80078-f376-4930-a9be-0752150efaed service placement] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:59:59.285815 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc80078-f376-4930-a9be-0752150efaed service placement] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:59:59.299396 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:59:59.343194 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e94446a-d43c-42b2-a29a-a657b85de5a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:59:59.343818 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e94446a-d43c-42b2-a29a-a657b85de5a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:59.344096 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e94446a-d43c-42b2-a29a-a657b85de5a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:59.344371 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e94446a-d43c-42b2-a29a-a657b85de5a2 admin admin] PATH_INFO: `/v3/projects/65676a714773423c8bf8a05c985c70ff` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:59.346182 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e94446a-d43c-42b2-a29a-a657b85de5a2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=65676a714773423c8bf8a05c985c70ff)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:59:59.348643 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e94446a-d43c-42b2-a29a-a657b85de5a2 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:59:59.394426 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-0e94446a-d43c-42b2-a29a-a657b85de5a2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '65676a714773423c8bf8a05c985c70ff', 'request_id': 'req-0e94446a-d43c-42b2-a29a-a657b85de5a2'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:59.400387 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-0e94446a-d43c-42b2-a29a-a657b85de5a2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '65676a714773423c8bf8a05c985c70ff', 'request_id': 'req-0e94446a-d43c-42b2-a29a-a657b85de5a2'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:59.407380 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-0e94446a-d43c-42b2-a29a-a657b85de5a2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '65676a714773423c8bf8a05c985c70ff', 'request_id': 'req-0e94446a-d43c-42b2-a29a-a657b85de5a2'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:59.418390 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-0e94446a-d43c-42b2-a29a-a657b85de5a2 admin admin] The token cache is being invalidate because project 65676a714773423c8bf8a05c985c70ff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81341) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:59:59.418674 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-0e94446a-d43c-42b2-a29a-a657b85de5a2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:59.419301 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-0e94446a-d43c-42b2-a29a-a657b85de5a2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:59.420939 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1062/2131] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Sep 1 14:59:59 2022] DELETE /identity/v3/projects/65676a714773423c8bf8a05c985c70ff => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:59:59.425050 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd60845-5578-46f2-a596-8171ab24f705 admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:59:59.426539 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-2dd60845-5578-46f2-a596-8171ab24f705 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:59:59.440453 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd60845-5578-46f2-a596-8171ab24f705 admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:59:59.502168 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5e688f-c433-4279-9c99-3bcdea6a110f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:59:59.502850 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5e688f-c433-4279-9c99-3bcdea6a110f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:59.503121 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5e688f-c433-4279-9c99-3bcdea6a110f admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:59.503384 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5e688f-c433-4279-9c99-3bcdea6a110f admin admin] PATH_INFO: `/v3/users/bd1b0d29e0924a129f35dbe5f0ddfcfd` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:59.504530 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5e688f-c433-4279-9c99-3bcdea6a110f admin admin] RBAC: Authorizing `identity:delete_user(user_id=bd1b0d29e0924a129f35dbe5f0ddfcfd)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:59:59.506499 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5e688f-c433-4279-9c99-3bcdea6a110f admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:59:59.557540 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-7c5e688f-c433-4279-9c99-3bcdea6a110f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bd1b0d29e0924a129f35dbe5f0ddfcfd'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:59.566125 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-7c5e688f-c433-4279-9c99-3bcdea6a110f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bd1b0d29e0924a129f35dbe5f0ddfcfd'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:59.566803 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-7c5e688f-c433-4279-9c99-3bcdea6a110f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bd1b0d29e0924a129f35dbe5f0ddfcfd'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:59.578537 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-7c5e688f-c433-4279-9c99-3bcdea6a110f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bd1b0d29e0924a129f35dbe5f0ddfcfd'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:59.579150 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-7c5e688f-c433-4279-9c99-3bcdea6a110f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bd1b0d29e0924a129f35dbe5f0ddfcfd'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:59.591878 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1070/2132] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Sep 1 14:59:59 2022] DELETE /identity/v3/users/bd1b0d29e0924a129f35dbe5f0ddfcfd => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 14:59:59.772637 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e94446a-d43c-42b2-a29a-a657b85de5a2 admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:59:59.774100 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-0e94446a-d43c-42b2-a29a-a657b85de5a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:59:59.787510 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e94446a-d43c-42b2-a29a-a657b85de5a2 admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:59:59.847211 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a05773-065d-4c44-8ed9-f42127196936 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:59:59.847841 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a05773-065d-4c44-8ed9-f42127196936 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:59.848120 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a05773-065d-4c44-8ed9-f42127196936 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:59.848395 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a05773-065d-4c44-8ed9-f42127196936 admin admin] PATH_INFO: `/v3/projects/5b73abfdf86a45e28ef866b625e15b1c` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:59.849550 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a05773-065d-4c44-8ed9-f42127196936 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5b73abfdf86a45e28ef866b625e15b1c)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:59:59.851596 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a05773-065d-4c44-8ed9-f42127196936 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:59:59.901785 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-e5a05773-065d-4c44-8ed9-f42127196936 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5b73abfdf86a45e28ef866b625e15b1c', 'request_id': 'req-e5a05773-065d-4c44-8ed9-f42127196936'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:59.902909 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:59:59.909627 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-e5a05773-065d-4c44-8ed9-f42127196936 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5b73abfdf86a45e28ef866b625e15b1c', 'request_id': 'req-e5a05773-065d-4c44-8ed9-f42127196936'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:59.917064 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-e5a05773-065d-4c44-8ed9-f42127196936 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5b73abfdf86a45e28ef866b625e15b1c', 'request_id': 'req-e5a05773-065d-4c44-8ed9-f42127196936'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:59.927821 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-e5a05773-065d-4c44-8ed9-f42127196936 admin admin] The token cache is being invalidate because project 5b73abfdf86a45e28ef866b625e15b1c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81341) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:59:59.928131 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-e5a05773-065d-4c44-8ed9-f42127196936 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:59.928763 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-e5a05773-065d-4c44-8ed9-f42127196936 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:59:59.934279 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1063/2133] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Sep 1 14:59:59 2022] DELETE /identity/v3/projects/5b73abfdf86a45e28ef866b625e15b1c => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:59:59.935931 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d7e17f-0033-43a9-98fe-4c7d8d57d25c None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:59:59.936422 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d7e17f-0033-43a9-98fe-4c7d8d57d25c None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:59:59.936549 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d7e17f-0033-43a9-98fe-4c7d8d57d25c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:59:59.956698 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-68d7e17f-0033-43a9-98fe-4c7d8d57d25c None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:59:59.957899 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-68d7e17f-0033-43a9-98fe-4c7d8d57d25c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:59:59.967130 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:DELETE=18,keystone:SELECT=80,keystone:INSERT=2 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 15:00:00.019462 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1071/2134] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 14:59:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 15:00:00.028873 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc651b4a-a783-4ab0-ab7c-b83541e999c8 None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:00:00.029294 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc651b4a-a783-4ab0-ab7c-b83541e999c8 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:00:00.029515 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc651b4a-a783-4ab0-ab7c-b83541e999c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:00:00.050694 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-cc651b4a-a783-4ab0-ab7c-b83541e999c8 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 15:00:00.052021 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-cc651b4a-a783-4ab0-ab7c-b83541e999c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:00:00.060085 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1064/2135] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 15:00:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 15:00:00.064989 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d7e17f-0033-43a9-98fe-4c7d8d57d25c None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:00:00.074780 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d7e17f-0033-43a9-98fe-4c7d8d57d25c None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:00:00.082975 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f643a3-2902-4804-b65f-8df42eec8f6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:00:00.083609 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f643a3-2902-4804-b65f-8df42eec8f6b admin admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:00:00.083890 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f643a3-2902-4804-b65f-8df42eec8f6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:00:00.084161 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f643a3-2902-4804-b65f-8df42eec8f6b admin admin] PATH_INFO: `/v3/domains` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:00:00.084833 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f643a3-2902-4804-b65f-8df42eec8f6b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 15:00:00.085136 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f643a3-2902-4804-b65f-8df42eec8f6b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:00:00.086785 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f643a3-2902-4804-b65f-8df42eec8f6b admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:00:00.102956 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1072/2136] 10.222.0.33 () {58 vars in 1153 bytes} [Thu Sep 1 15:00:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 15:00:00.104121 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc651b4a-a783-4ab0-ab7c-b83541e999c8 None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:00:00.114036 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc651b4a-a783-4ab0-ab7c-b83541e999c8 None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:00:00.118775 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78b0085-a3b6-40d9-bc80-80473215c47e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:00:00.119373 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78b0085-a3b6-40d9-bc80-80473215c47e admin admin] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:00:00.119652 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78b0085-a3b6-40d9-bc80-80473215c47e admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:00:00.120244 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78b0085-a3b6-40d9-bc80-80473215c47e admin admin] PATH_INFO: `/v3/projects` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:00:00.120548 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78b0085-a3b6-40d9-bc80-80473215c47e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:00:00.122511 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78b0085-a3b6-40d9-bc80-80473215c47e admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:00:00.155502 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1065/2137] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Sep 1 15:00:00 2022] POST /identity/v3/projects => generated 363 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 15:00:00.159707 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f643a3-2902-4804-b65f-8df42eec8f6b admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:00:00.169631 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f643a3-2902-4804-b65f-8df42eec8f6b admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:00:00.229372 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd745a6-0691-46e7-9aa3-460a3cdbbe45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:00:00.230242 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd745a6-0691-46e7-9aa3-460a3cdbbe45 admin admin] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:00:00.230543 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd745a6-0691-46e7-9aa3-460a3cdbbe45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:00:00.230815 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd745a6-0691-46e7-9aa3-460a3cdbbe45 admin admin] PATH_INFO: `/v3/users` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:00:00.231466 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd745a6-0691-46e7-9aa3-460a3cdbbe45 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:00:00.233366 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd745a6-0691-46e7-9aa3-460a3cdbbe45 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:00:00.254390 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1073/2138] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Sep 1 15:00:00 2022] POST /identity/v3/users => generated 282 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 15:00:00.258486 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78b0085-a3b6-40d9-bc80-80473215c47e admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:00:00.268606 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78b0085-a3b6-40d9-bc80-80473215c47e admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:00:00.273074 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c230429-cc82-4df6-ac98-85a9af1bf071 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:00:00.273658 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c230429-cc82-4df6-ac98-85a9af1bf071 admin admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:00:00.273931 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c230429-cc82-4df6-ac98-85a9af1bf071 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:00:00.274197 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c230429-cc82-4df6-ac98-85a9af1bf071 admin admin] PATH_INFO: `/v3/roles` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:00:00.274861 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c230429-cc82-4df6-ac98-85a9af1bf071 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:00:00.276449 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c230429-cc82-4df6-ac98-85a9af1bf071 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:00:00.286118 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1066/2139] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 1 15:00:00 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 15:00:00.290084 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd745a6-0691-46e7-9aa3-460a3cdbbe45 admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:00:00.301147 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd745a6-0691-46e7-9aa3-460a3cdbbe45 admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:00:00.306364 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62280e1-835d-4e63-bd9b-ba188724b1ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:00:00.306971 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62280e1-835d-4e63-bd9b-ba188724b1ab admin admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:00:00.307250 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62280e1-835d-4e63-bd9b-ba188724b1ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:00:00.307515 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62280e1-835d-4e63-bd9b-ba188724b1ab admin admin] PATH_INFO: `/v3/roles` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:00:00.308165 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62280e1-835d-4e63-bd9b-ba188724b1ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:00:00.309765 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62280e1-835d-4e63-bd9b-ba188724b1ab admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:00:00.321284 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1074/2140] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 1 15:00:00 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 15:00:00.325586 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c230429-cc82-4df6-ac98-85a9af1bf071 admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:00:00.335537 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c230429-cc82-4df6-ac98-85a9af1bf071 admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:00:00.340253 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813160c3-bc6e-421c-ab33-d97506b14260 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:00:00.340910 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813160c3-bc6e-421c-ab33-d97506b14260 admin admin] REQUEST_METHOD: `PUT` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:00:00.341178 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813160c3-bc6e-421c-ab33-d97506b14260 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:00:00.341447 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813160c3-bc6e-421c-ab33-d97506b14260 admin admin] PATH_INFO: `/v3/projects/1e4f0531afa4445ca94e5c05d05dca32/users/9d833fcb0e4144b19666120a183cea03/roles/5d1feef0e13d4874928bd86258c4e3b2` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:00:00.356931 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813160c3-bc6e-421c-ab33-d97506b14260 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e4f0531afa4445ca94e5c05d05dca32, user_id=9d833fcb0e4144b19666120a183cea03, role_id=5d1feef0e13d4874928bd86258c4e3b2)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:00:00.359646 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813160c3-bc6e-421c-ab33-d97506b14260 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:00:00.372879 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1067/2141] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 1 15:00:00 2022] PUT /identity/v3/projects/1e4f0531afa4445ca94e5c05d05dca32/users/9d833fcb0e4144b19666120a183cea03/roles/5d1feef0e13d4874928bd86258c4e3b2 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 15:00:00.383690 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feaab904-3a1d-46b5-a8d8-8eed342a80d8 None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:00:00.383974 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feaab904-3a1d-46b5-a8d8-8eed342a80d8 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:00:00.384241 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feaab904-3a1d-46b5-a8d8-8eed342a80d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:00:00.413546 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-feaab904-3a1d-46b5-a8d8-8eed342a80d8 None None] MFA Rules not processed for user `9d833fcb0e4144b19666120a183cea03`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 15:00:00.414791 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-feaab904-3a1d-46b5-a8d8-8eed342a80d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:00:00.481600 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1075/2142] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 15:00:00 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 15:00:00.486523 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813160c3-bc6e-421c-ab33-d97506b14260 admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:00:00.550357 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813160c3-bc6e-421c-ab33-d97506b14260 admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:00:00.555306 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb52aec8-03be-45e5-a4c9-3a4f7cceb496 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:00:00.555902 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb52aec8-03be-45e5-a4c9-3a4f7cceb496 admin admin] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:00:00.556218 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb52aec8-03be-45e5-a4c9-3a4f7cceb496 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:00:00.556488 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb52aec8-03be-45e5-a4c9-3a4f7cceb496 admin admin] PATH_INFO: `/v3/projects` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:00:00.557123 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb52aec8-03be-45e5-a4c9-3a4f7cceb496 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:00:00.559069 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb52aec8-03be-45e5-a4c9-3a4f7cceb496 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:00:00.584297 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1068/2143] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Sep 1 15:00:00 2022] POST /identity/v3/projects => generated 363 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 15:00:00.588336 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feaab904-3a1d-46b5-a8d8-8eed342a80d8 None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:00:00.599847 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feaab904-3a1d-46b5-a8d8-8eed342a80d8 None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:00:00.607921 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 15:00:00.658976 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3312e5-5cd0-4a1a-bd74-c40c5abd654d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:00:00.659616 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3312e5-5cd0-4a1a-bd74-c40c5abd654d admin admin] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:00:00.659896 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3312e5-5cd0-4a1a-bd74-c40c5abd654d admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:00:00.660166 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3312e5-5cd0-4a1a-bd74-c40c5abd654d admin admin] PATH_INFO: `/v3/users` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:00:00.660797 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3312e5-5cd0-4a1a-bd74-c40c5abd654d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:00:00.662862 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3312e5-5cd0-4a1a-bd74-c40c5abd654d admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:00:00.685142 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1076/2144] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Sep 1 15:00:00 2022] POST /identity/v3/users => generated 288 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 15:00:00.689213 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb52aec8-03be-45e5-a4c9-3a4f7cceb496 admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:00:00.699403 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb52aec8-03be-45e5-a4c9-3a4f7cceb496 admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:00:00.703951 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f42d1f1-68d1-4bd9-9ec0-40ce98c7a059 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:00:00.704531 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f42d1f1-68d1-4bd9-9ec0-40ce98c7a059 admin admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:00:00.704824 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f42d1f1-68d1-4bd9-9ec0-40ce98c7a059 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:00:00.705093 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f42d1f1-68d1-4bd9-9ec0-40ce98c7a059 admin admin] PATH_INFO: `/v3/roles` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:00:00.705720 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f42d1f1-68d1-4bd9-9ec0-40ce98c7a059 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:00:00.707447 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f42d1f1-68d1-4bd9-9ec0-40ce98c7a059 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:00:00.718628 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1069/2145] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 1 15:00:00 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 15:00:00.725279 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3312e5-5cd0-4a1a-bd74-c40c5abd654d admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:00:00.736657 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3312e5-5cd0-4a1a-bd74-c40c5abd654d admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:00:00.741192 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baef1f59-7dd4-459f-9e32-0169f38161a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:00:00.741843 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baef1f59-7dd4-459f-9e32-0169f38161a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:00:00.742492 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baef1f59-7dd4-459f-9e32-0169f38161a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:00:00.742765 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baef1f59-7dd4-459f-9e32-0169f38161a3 admin admin] PATH_INFO: `/v3/projects/baa482d008a3490abcc0bd66fe3c4aaa/users/ccb48b59655e40caaa6fb83b7986cd8a/roles/2a2adb769e55476b82e5014897e0a9c0` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:00:00.757839 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baef1f59-7dd4-459f-9e32-0169f38161a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=baa482d008a3490abcc0bd66fe3c4aaa, user_id=ccb48b59655e40caaa6fb83b7986cd8a, role_id=2a2adb769e55476b82e5014897e0a9c0)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:00:00.761366 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baef1f59-7dd4-459f-9e32-0169f38161a3 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:00:00.775826 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1077/2146] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 1 15:00:00 2022] PUT /identity/v3/projects/baa482d008a3490abcc0bd66fe3c4aaa/users/ccb48b59655e40caaa6fb83b7986cd8a/roles/2a2adb769e55476b82e5014897e0a9c0 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 15:00:00.779954 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f42d1f1-68d1-4bd9-9ec0-40ce98c7a059 admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:00:00.790275 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f42d1f1-68d1-4bd9-9ec0-40ce98c7a059 admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:00:00.834099 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 15:00:00.848635 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1366d7f-9666-4a22-9453-493093e65949 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:00:00.849260 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1366d7f-9666-4a22-9453-493093e65949 admin admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:00:00.849550 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1366d7f-9666-4a22-9453-493093e65949 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:00:00.850051 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1366d7f-9666-4a22-9453-493093e65949 admin admin] PATH_INFO: `/v3/roles` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:00:00.850817 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1366d7f-9666-4a22-9453-493093e65949 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:00:00.852650 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1366d7f-9666-4a22-9453-493093e65949 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:00:00.863009 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1070/2147] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 1 15:00:00 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 15:00:00.867309 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baef1f59-7dd4-459f-9e32-0169f38161a3 admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:00:00.877267 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baef1f59-7dd4-459f-9e32-0169f38161a3 admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:00:00.881842 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a140ef0-53e8-4e74-bdfa-effee07e2441 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:00:00.882502 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a140ef0-53e8-4e74-bdfa-effee07e2441 admin admin] REQUEST_METHOD: `PUT` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:00:00.882903 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a140ef0-53e8-4e74-bdfa-effee07e2441 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:00:00.883031 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a140ef0-53e8-4e74-bdfa-effee07e2441 admin admin] PATH_INFO: `/v3/projects/baa482d008a3490abcc0bd66fe3c4aaa/users/ccb48b59655e40caaa6fb83b7986cd8a/roles/2a2adb769e55476b82e5014897e0a9c0` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:00:00.884922 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a140ef0-53e8-4e74-bdfa-effee07e2441 admin admin] RBAC: Authorizing `identity:create_grant(project_id=baa482d008a3490abcc0bd66fe3c4aaa, user_id=ccb48b59655e40caaa6fb83b7986cd8a, role_id=2a2adb769e55476b82e5014897e0a9c0)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:00:00.887650 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a140ef0-53e8-4e74-bdfa-effee07e2441 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:00:00.898768 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1078/2148] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 1 15:00:00 2022] PUT /identity/v3/projects/baa482d008a3490abcc0bd66fe3c4aaa/users/ccb48b59655e40caaa6fb83b7986cd8a/roles/2a2adb769e55476b82e5014897e0a9c0 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 15:00:00.910730 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afe6ecf-f5cf-44f2-999f-24cec3b2bc33 None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:00:00.911172 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afe6ecf-f5cf-44f2-999f-24cec3b2bc33 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:00:00.911295 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afe6ecf-f5cf-44f2-999f-24cec3b2bc33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:00:00.939568 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-4afe6ecf-f5cf-44f2-999f-24cec3b2bc33 None None] MFA Rules not processed for user `ccb48b59655e40caaa6fb83b7986cd8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 15:00:00.940822 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-4afe6ecf-f5cf-44f2-999f-24cec3b2bc33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:00:01.008431 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1071/2149] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 15:00:00 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 15:00:01.020050 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a140ef0-53e8-4e74-bdfa-effee07e2441 admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:00:01.021832 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-7a140ef0-53e8-4e74-bdfa-effee07e2441 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:00:01.081674 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a140ef0-53e8-4e74-bdfa-effee07e2441 admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:00:01.085264 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d79411f-3fd1-44fd-8fa8-fe890f043239 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3b81739e34e4e648f88b6d73fcb5e46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:00:01.085863 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d79411f-3fd1-44fd-8fa8-fe890f043239 service cinder] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:00:01.086139 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d79411f-3fd1-44fd-8fa8-fe890f043239 service cinder] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:00:01.086435 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d79411f-3fd1-44fd-8fa8-fe890f043239 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:00:01.095845 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d79411f-3fd1-44fd-8fa8-fe890f043239 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:00:01.097523 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d79411f-3fd1-44fd-8fa8-fe890f043239 service cinder] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:00:01.106838 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1079/2150] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 15:00:01 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 15:00:01.182464 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afe6ecf-f5cf-44f2-999f-24cec3b2bc33 None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:00:01.193213 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afe6ecf-f5cf-44f2-999f-24cec3b2bc33 None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:00:01.196895 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0743ea8-8ab9-4a08-abe0-335c056c223f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3b81739e34e4e648f88b6d73fcb5e46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:00:01.197498 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0743ea8-8ab9-4a08-abe0-335c056c223f service cinder] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:00:01.197775 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0743ea8-8ab9-4a08-abe0-335c056c223f service cinder] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:00:01.198049 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0743ea8-8ab9-4a08-abe0-335c056c223f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:00:01.257850 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0743ea8-8ab9-4a08-abe0-335c056c223f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:00:01.259390 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0743ea8-8ab9-4a08-abe0-335c056c223f service cinder] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:00:01.268910 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1072/2151] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 15:00:01 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 15:00:11.102597 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=65,keystone:INSERT=5 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 15:00:11.264211 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=69 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 15:00:14.843681 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d79411f-3fd1-44fd-8fa8-fe890f043239 service cinder] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:00:14.904450 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d79411f-3fd1-44fd-8fa8-fe890f043239 service cinder] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:00:14.909004 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cc7c76-1f89-4d3e-a8d0-9e6a4535b60f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:00:14.909616 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cc7c76-1f89-4d3e-a8d0-9e6a4535b60f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:00:14.909889 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cc7c76-1f89-4d3e-a8d0-9e6a4535b60f admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:00:14.910152 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cc7c76-1f89-4d3e-a8d0-9e6a4535b60f admin admin] PATH_INFO: `/v3/users/9d833fcb0e4144b19666120a183cea03` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:00:14.911289 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97cc7c76-1f89-4d3e-a8d0-9e6a4535b60f admin admin] RBAC: Authorizing `identity:delete_user(user_id=9d833fcb0e4144b19666120a183cea03)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:00:14.913256 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97cc7c76-1f89-4d3e-a8d0-9e6a4535b60f admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:00:14.956858 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-97cc7c76-1f89-4d3e-a8d0-9e6a4535b60f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9d833fcb0e4144b19666120a183cea03'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:00:14.963629 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-97cc7c76-1f89-4d3e-a8d0-9e6a4535b60f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9d833fcb0e4144b19666120a183cea03'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:00:14.964255 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-97cc7c76-1f89-4d3e-a8d0-9e6a4535b60f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9d833fcb0e4144b19666120a183cea03'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:00:14.977639 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-97cc7c76-1f89-4d3e-a8d0-9e6a4535b60f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9d833fcb0e4144b19666120a183cea03'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:00:14.978437 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-97cc7c76-1f89-4d3e-a8d0-9e6a4535b60f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9d833fcb0e4144b19666120a183cea03'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:00:14.991931 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1080/2152] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Sep 1 15:00:14 2022] DELETE /identity/v3/users/9d833fcb0e4144b19666120a183cea03 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 15:00:14.996352 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089ceb2e-02c8-48d9-ab3b-c98434c4cfe9 None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:00:14.996829 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089ceb2e-02c8-48d9-ab3b-c98434c4cfe9 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:00:14.996949 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089ceb2e-02c8-48d9-ab3b-c98434c4cfe9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:00:15.017613 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-089ceb2e-02c8-48d9-ab3b-c98434c4cfe9 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 15:00:15.018889 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-089ceb2e-02c8-48d9-ab3b-c98434c4cfe9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:00:15.079782 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1073/2153] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 15:00:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 15:00:15.090145 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cc7c76-1f89-4d3e-a8d0-9e6a4535b60f admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:00:15.091956 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-97cc7c76-1f89-4d3e-a8d0-9e6a4535b60f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:00:15.151790 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cc7c76-1f89-4d3e-a8d0-9e6a4535b60f admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:00:15.155456 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce3c537-4658-4b76-a047-81a1378da95a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2790149639aa418bbc4e4982e93eda44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:00:15.156062 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce3c537-4658-4b76-a047-81a1378da95a service neutron] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:00:15.156344 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce3c537-4658-4b76-a047-81a1378da95a service neutron] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:00:15.156614 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce3c537-4658-4b76-a047-81a1378da95a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:00:15.167124 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce3c537-4658-4b76-a047-81a1378da95a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:00:15.169985 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce3c537-4658-4b76-a047-81a1378da95a service neutron] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:00:15.180867 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1081/2154] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 15:00:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 15:00:15.477361 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089ceb2e-02c8-48d9-ab3b-c98434c4cfe9 None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:00:15.478888 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-089ceb2e-02c8-48d9-ab3b-c98434c4cfe9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:00:15.491726 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089ceb2e-02c8-48d9-ab3b-c98434c4cfe9 None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:00:15.497725 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c39edfb-9bb9-4a38-bd96-40421e8d4d1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:00:15.498452 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c39edfb-9bb9-4a38-bd96-40421e8d4d1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:00:15.498742 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c39edfb-9bb9-4a38-bd96-40421e8d4d1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:00:15.499102 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c39edfb-9bb9-4a38-bd96-40421e8d4d1b admin admin] PATH_INFO: `/v3/projects/1e4f0531afa4445ca94e5c05d05dca32` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:00:15.500500 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c39edfb-9bb9-4a38-bd96-40421e8d4d1b admin admin] RBAC: Authorizing `identity:delete_project(project_id=1e4f0531afa4445ca94e5c05d05dca32)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:00:15.504341 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c39edfb-9bb9-4a38-bd96-40421e8d4d1b admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:00:15.553572 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-0c39edfb-9bb9-4a38-bd96-40421e8d4d1b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1e4f0531afa4445ca94e5c05d05dca32', 'request_id': 'req-0c39edfb-9bb9-4a38-bd96-40421e8d4d1b'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:00:15.559692 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-0c39edfb-9bb9-4a38-bd96-40421e8d4d1b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1e4f0531afa4445ca94e5c05d05dca32', 'request_id': 'req-0c39edfb-9bb9-4a38-bd96-40421e8d4d1b'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:00:15.568021 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-0c39edfb-9bb9-4a38-bd96-40421e8d4d1b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1e4f0531afa4445ca94e5c05d05dca32', 'request_id': 'req-0c39edfb-9bb9-4a38-bd96-40421e8d4d1b'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:00:15.578721 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-0c39edfb-9bb9-4a38-bd96-40421e8d4d1b admin admin] The token cache is being invalidate because project 1e4f0531afa4445ca94e5c05d05dca32 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81341) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 15:00:15.579011 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-0c39edfb-9bb9-4a38-bd96-40421e8d4d1b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:00:15.579637 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-0c39edfb-9bb9-4a38-bd96-40421e8d4d1b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:00:15.581127 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1074/2155] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Sep 1 15:00:15 2022] DELETE /identity/v3/projects/1e4f0531afa4445ca94e5c05d05dca32 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 15:00:15.585345 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce3c537-4658-4b76-a047-81a1378da95a service neutron] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:00:15.595136 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce3c537-4658-4b76-a047-81a1378da95a service neutron] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:00:15.613132 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 15:00:15.654543 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18c0993-e26e-4361-a4c9-06e20785018a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:00:15.655328 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18c0993-e26e-4361-a4c9-06e20785018a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:00:15.655631 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18c0993-e26e-4361-a4c9-06e20785018a admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:00:15.655906 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18c0993-e26e-4361-a4c9-06e20785018a admin admin] PATH_INFO: `/v3/users/ccb48b59655e40caaa6fb83b7986cd8a` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:00:15.657043 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18c0993-e26e-4361-a4c9-06e20785018a admin admin] RBAC: Authorizing `identity:delete_user(user_id=ccb48b59655e40caaa6fb83b7986cd8a)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:00:15.661157 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18c0993-e26e-4361-a4c9-06e20785018a admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:00:15.718272 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-f18c0993-e26e-4361-a4c9-06e20785018a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ccb48b59655e40caaa6fb83b7986cd8a'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:00:15.727097 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-f18c0993-e26e-4361-a4c9-06e20785018a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ccb48b59655e40caaa6fb83b7986cd8a'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:00:15.727699 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-f18c0993-e26e-4361-a4c9-06e20785018a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ccb48b59655e40caaa6fb83b7986cd8a'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:00:15.740014 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-f18c0993-e26e-4361-a4c9-06e20785018a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ccb48b59655e40caaa6fb83b7986cd8a'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:00:15.740902 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-f18c0993-e26e-4361-a4c9-06e20785018a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ccb48b59655e40caaa6fb83b7986cd8a'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:00:15.760191 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1082/2156] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Sep 1 15:00:15 2022] DELETE /identity/v3/users/ccb48b59655e40caaa6fb83b7986cd8a => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 15:00:16.111094 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c39edfb-9bb9-4a38-bd96-40421e8d4d1b admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:00:16.112947 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-0c39edfb-9bb9-4a38-bd96-40421e8d4d1b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:00:16.138585 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c39edfb-9bb9-4a38-bd96-40421e8d4d1b admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:00:16.214564 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54b52a3-ff02-483e-b492-a49a7087f5ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:00:16.215217 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54b52a3-ff02-483e-b492-a49a7087f5ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:00:16.215493 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54b52a3-ff02-483e-b492-a49a7087f5ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:00:16.215762 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54b52a3-ff02-483e-b492-a49a7087f5ac admin admin] PATH_INFO: `/v3/projects/baa482d008a3490abcc0bd66fe3c4aaa` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:00:16.216923 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54b52a3-ff02-483e-b492-a49a7087f5ac admin admin] RBAC: Authorizing `identity:delete_project(project_id=baa482d008a3490abcc0bd66fe3c4aaa)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:00:16.219081 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54b52a3-ff02-483e-b492-a49a7087f5ac admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:00:16.254175 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 15:00:16.274355 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-e54b52a3-ff02-483e-b492-a49a7087f5ac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'baa482d008a3490abcc0bd66fe3c4aaa', 'request_id': 'req-e54b52a3-ff02-483e-b492-a49a7087f5ac'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:00:16.281469 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-e54b52a3-ff02-483e-b492-a49a7087f5ac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'baa482d008a3490abcc0bd66fe3c4aaa', 'request_id': 'req-e54b52a3-ff02-483e-b492-a49a7087f5ac'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:00:16.290015 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-e54b52a3-ff02-483e-b492-a49a7087f5ac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'baa482d008a3490abcc0bd66fe3c4aaa', 'request_id': 'req-e54b52a3-ff02-483e-b492-a49a7087f5ac'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:00:16.306805 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-e54b52a3-ff02-483e-b492-a49a7087f5ac admin admin] The token cache is being invalidate because project baa482d008a3490abcc0bd66fe3c4aaa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81341) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 15:00:16.307111 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-e54b52a3-ff02-483e-b492-a49a7087f5ac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:00:16.307737 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-e54b52a3-ff02-483e-b492-a49a7087f5ac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:00:16.309589 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1075/2157] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Sep 1 15:00:16 2022] DELETE /identity/v3/projects/baa482d008a3490abcc0bd66fe3c4aaa => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 15:00:16.315328 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b42060a-456e-4655-a919-2d1e6e4ba9fa None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:00:16.315328 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b42060a-456e-4655-a919-2d1e6e4ba9fa None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:00:16.315328 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b42060a-456e-4655-a919-2d1e6e4ba9fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:00:16.339022 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-9b42060a-456e-4655-a919-2d1e6e4ba9fa None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 15:00:16.340249 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-9b42060a-456e-4655-a919-2d1e6e4ba9fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:00:16.351343 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54b52a3-ff02-483e-b492-a49a7087f5ac admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:00:16.352903 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-e54b52a3-ff02-483e-b492-a49a7087f5ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:00:16.428747 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1083/2158] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 15:00:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 15:00:16.434869 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54b52a3-ff02-483e-b492-a49a7087f5ac admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:00:16.436207 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0573f61-6174-4830-a448-06a6ab72a900 None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:00:16.436498 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0573f61-6174-4830-a448-06a6ab72a900 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:00:16.436768 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0573f61-6174-4830-a448-06a6ab72a900 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:00:16.440181 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afc0011-f40b-42de-abfa-d7744c5ff7d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:00:16.440810 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afc0011-f40b-42de-abfa-d7744c5ff7d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:00:16.441093 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afc0011-f40b-42de-abfa-d7744c5ff7d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:00:16.441368 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afc0011-f40b-42de-abfa-d7744c5ff7d3 admin admin] PATH_INFO: `/v3/users/ee1ea25915c44868a4e813a471abfc1c` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:00:16.442560 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afc0011-f40b-42de-abfa-d7744c5ff7d3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ee1ea25915c44868a4e813a471abfc1c)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:00:16.444729 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afc0011-f40b-42de-abfa-d7744c5ff7d3 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:00:16.457273 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-d0573f61-6174-4830-a448-06a6ab72a900 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 15:00:16.459173 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-d0573f61-6174-4830-a448-06a6ab72a900 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:00:16.470404 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1084/2159] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 15:00:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 15:00:16.473423 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0573f61-6174-4830-a448-06a6ab72a900 None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:00:16.483741 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0573f61-6174-4830-a448-06a6ab72a900 None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:00:16.488643 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c04a24d-8e71-4cf2-8357-7eb637ffb2ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:00:16.489243 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c04a24d-8e71-4cf2-8357-7eb637ffb2ea admin admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:00:16.489686 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c04a24d-8e71-4cf2-8357-7eb637ffb2ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:00:16.489967 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c04a24d-8e71-4cf2-8357-7eb637ffb2ea admin admin] PATH_INFO: `/v3/domains` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:00:16.490635 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c04a24d-8e71-4cf2-8357-7eb637ffb2ea admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 15:00:16.490943 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c04a24d-8e71-4cf2-8357-7eb637ffb2ea admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:00:16.492646 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c04a24d-8e71-4cf2-8357-7eb637ffb2ea admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:00:16.508637 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1085/2160] 10.222.0.33 () {58 vars in 1153 bytes} [Thu Sep 1 15:00:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 15:00:16.509847 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c04a24d-8e71-4cf2-8357-7eb637ffb2ea admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:00:16.511626 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-1afc0011-f40b-42de-abfa-d7744c5ff7d3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ee1ea25915c44868a4e813a471abfc1c'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:00:16.520067 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c04a24d-8e71-4cf2-8357-7eb637ffb2ea admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:00:16.521385 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-1afc0011-f40b-42de-abfa-d7744c5ff7d3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ee1ea25915c44868a4e813a471abfc1c'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:00:16.522117 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-1afc0011-f40b-42de-abfa-d7744c5ff7d3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ee1ea25915c44868a4e813a471abfc1c'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:00:16.531037 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541b930a-bf27-40c2-b722-4788cdb4b410 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:00:16.532333 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541b930a-bf27-40c2-b722-4788cdb4b410 admin admin] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:00:16.532778 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541b930a-bf27-40c2-b722-4788cdb4b410 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:00:16.533058 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541b930a-bf27-40c2-b722-4788cdb4b410 admin admin] PATH_INFO: `/v3/projects` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:00:16.533707 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541b930a-bf27-40c2-b722-4788cdb4b410 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:00:16.536151 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541b930a-bf27-40c2-b722-4788cdb4b410 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:00:16.537125 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-1afc0011-f40b-42de-abfa-d7744c5ff7d3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ee1ea25915c44868a4e813a471abfc1c'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:00:16.538001 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-1afc0011-f40b-42de-abfa-d7744c5ff7d3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ee1ea25915c44868a4e813a471abfc1c'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:00:16.555777 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1076/2161] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Sep 1 15:00:16 2022] DELETE /identity/v3/users/ee1ea25915c44868a4e813a471abfc1c => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 15:00:16.557982 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8429b3e-f911-4500-b55b-88892e4f1c7b None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:00:16.558143 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8429b3e-f911-4500-b55b-88892e4f1c7b None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:00:16.558320 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8429b3e-f911-4500-b55b-88892e4f1c7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:00:16.568626 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 15:00:16.578397 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1086/2162] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Sep 1 15:00:16 2022] POST /identity/v3/projects => generated 363 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 15:00:16.581315 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-f8429b3e-f911-4500-b55b-88892e4f1c7b None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 15:00:16.581740 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541b930a-bf27-40c2-b722-4788cdb4b410 admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:00:16.582564 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-f8429b3e-f911-4500-b55b-88892e4f1c7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:00:16.584002 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-541b930a-bf27-40c2-b722-4788cdb4b410 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:00:16.615258 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:DELETE=16,keystone:SELECT=82,keystone:INSERT=2 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 15:00:16.652146 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1077/2163] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 15:00:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 15:00:16.669178 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541b930a-bf27-40c2-b722-4788cdb4b410 admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:00:16.671679 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8429b3e-f911-4500-b55b-88892e4f1c7b None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:00:16.673153 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-f8429b3e-f911-4500-b55b-88892e4f1c7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:00:16.674514 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fce791-33b4-468b-acda-b784889e7a46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:00:16.675132 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fce791-33b4-468b-acda-b784889e7a46 admin admin] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:00:16.675414 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fce791-33b4-468b-acda-b784889e7a46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:00:16.675685 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fce791-33b4-468b-acda-b784889e7a46 admin admin] PATH_INFO: `/v3/users` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:00:16.676325 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82fce791-33b4-468b-acda-b784889e7a46 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:00:16.679087 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82fce791-33b4-468b-acda-b784889e7a46 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:00:16.705598 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1087/2164] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Sep 1 15:00:16 2022] POST /identity/v3/users => generated 282 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 15:00:16.709889 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fce791-33b4-468b-acda-b784889e7a46 admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:00:16.721578 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fce791-33b4-468b-acda-b784889e7a46 admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:00:16.726874 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0c383b-5c3e-4510-ab4e-3ec73480d8d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:00:16.727556 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0c383b-5c3e-4510-ab4e-3ec73480d8d7 admin admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:00:16.728390 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0c383b-5c3e-4510-ab4e-3ec73480d8d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:00:16.728743 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0c383b-5c3e-4510-ab4e-3ec73480d8d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:00:16.729464 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0c383b-5c3e-4510-ab4e-3ec73480d8d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:00:16.731231 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0c383b-5c3e-4510-ab4e-3ec73480d8d7 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:00:16.738446 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8429b3e-f911-4500-b55b-88892e4f1c7b None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:00:16.742817 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1088/2165] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 1 15:00:16 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 15:00:16.744637 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a31e9a-8338-4294-aea6-f7866d2fcd4f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2790149639aa418bbc4e4982e93eda44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:00:16.745250 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a31e9a-8338-4294-aea6-f7866d2fcd4f service neutron] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:00:16.745685 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a31e9a-8338-4294-aea6-f7866d2fcd4f service neutron] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:00:16.746284 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a31e9a-8338-4294-aea6-f7866d2fcd4f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:00:16.746872 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0c383b-5c3e-4510-ab4e-3ec73480d8d7 admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:00:16.757757 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a31e9a-8338-4294-aea6-f7866d2fcd4f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:00:16.759428 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a31e9a-8338-4294-aea6-f7866d2fcd4f service neutron] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:00:16.763892 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0c383b-5c3e-4510-ab4e-3ec73480d8d7 admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:00:16.770115 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1078/2166] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 15:00:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 15:00:16.770309 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd2029b-c3ed-4f63-b153-3278da98ecef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:00:16.771474 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd2029b-c3ed-4f63-b153-3278da98ecef admin admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:00:16.771884 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd2029b-c3ed-4f63-b153-3278da98ecef admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:00:16.772163 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd2029b-c3ed-4f63-b153-3278da98ecef admin admin] PATH_INFO: `/v3/roles` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:00:16.772817 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd2029b-c3ed-4f63-b153-3278da98ecef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:00:16.774446 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd2029b-c3ed-4f63-b153-3278da98ecef admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:00:16.785700 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1089/2167] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 1 15:00:16 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 15:00:16.790463 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a31e9a-8338-4294-aea6-f7866d2fcd4f service neutron] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:00:16.804503 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a31e9a-8338-4294-aea6-f7866d2fcd4f service neutron] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:00:16.810409 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed462df1-5799-4a43-9d71-596a6f5d2175 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:00:16.811049 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed462df1-5799-4a43-9d71-596a6f5d2175 admin admin] REQUEST_METHOD: `PUT` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:00:16.811503 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed462df1-5799-4a43-9d71-596a6f5d2175 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:00:16.811781 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed462df1-5799-4a43-9d71-596a6f5d2175 admin admin] PATH_INFO: `/v3/projects/728696de203440d38cd68ae50c3cbde5/users/67eea39c1f794dd89a51c15b1f46c593/roles/5d1feef0e13d4874928bd86258c4e3b2` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:00:16.828776 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed462df1-5799-4a43-9d71-596a6f5d2175 admin admin] RBAC: Authorizing `identity:create_grant(project_id=728696de203440d38cd68ae50c3cbde5, user_id=67eea39c1f794dd89a51c15b1f46c593, role_id=5d1feef0e13d4874928bd86258c4e3b2)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:00:16.831687 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed462df1-5799-4a43-9d71-596a6f5d2175 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:00:16.846830 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1079/2168] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 1 15:00:16 2022] PUT /identity/v3/projects/728696de203440d38cd68ae50c3cbde5/users/67eea39c1f794dd89a51c15b1f46c593/roles/5d1feef0e13d4874928bd86258c4e3b2 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 15:00:16.858005 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350ba637-9c49-4440-9d91-804b8e4250c2 None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:00:16.858364 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350ba637-9c49-4440-9d91-804b8e4250c2 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:00:16.858639 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350ba637-9c49-4440-9d91-804b8e4250c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:00:16.889301 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-350ba637-9c49-4440-9d91-804b8e4250c2 None None] MFA Rules not processed for user `67eea39c1f794dd89a51c15b1f46c593`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 15:00:16.890551 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-350ba637-9c49-4440-9d91-804b8e4250c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:00:16.981764 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1090/2169] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 15:00:16 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 15:00:16.986080 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed462df1-5799-4a43-9d71-596a6f5d2175 admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:00:17.050141 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350ba637-9c49-4440-9d91-804b8e4250c2 None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:00:17.051689 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-350ba637-9c49-4440-9d91-804b8e4250c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:00:17.061574 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed462df1-5799-4a43-9d71-596a6f5d2175 admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:00:17.065723 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350ba637-9c49-4440-9d91-804b8e4250c2 None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:00:17.071774 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c150fbfa-a788-4f9e-8074-1077f1410ee2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:00:17.071774 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c150fbfa-a788-4f9e-8074-1077f1410ee2 admin admin] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:00:17.071774 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c150fbfa-a788-4f9e-8074-1077f1410ee2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:00:17.071774 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c150fbfa-a788-4f9e-8074-1077f1410ee2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:00:17.072245 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c150fbfa-a788-4f9e-8074-1077f1410ee2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:00:17.072245 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c150fbfa-a788-4f9e-8074-1077f1410ee2 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:00:17.076951 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1ad391-f401-4beb-bc9c-994e4ab14ce1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:00:17.077990 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1ad391-f401-4beb-bc9c-994e4ab14ce1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:00:17.078415 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1ad391-f401-4beb-bc9c-994e4ab14ce1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:00:17.078761 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1ad391-f401-4beb-bc9c-994e4ab14ce1 admin admin] PATH_INFO: `/v3/users/5710749db2ed489fb3ce3b4cc8c9cf8a` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:00:17.080154 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1ad391-f401-4beb-bc9c-994e4ab14ce1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5710749db2ed489fb3ce3b4cc8c9cf8a)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:00:17.082772 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1ad391-f401-4beb-bc9c-994e4ab14ce1 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:00:17.101730 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1080/2170] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Sep 1 15:00:16 2022] POST /identity/v3/projects => generated 361 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 15:00:17.105435 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c150fbfa-a788-4f9e-8074-1077f1410ee2 admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:00:17.106873 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-c150fbfa-a788-4f9e-8074-1077f1410ee2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:00:17.147709 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-8c1ad391-f401-4beb-bc9c-994e4ab14ce1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5710749db2ed489fb3ce3b4cc8c9cf8a'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:00:17.151834 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 15:00:17.153218 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 15:00:17.156611 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-8c1ad391-f401-4beb-bc9c-994e4ab14ce1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5710749db2ed489fb3ce3b4cc8c9cf8a'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:00:17.157450 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-8c1ad391-f401-4beb-bc9c-994e4ab14ce1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5710749db2ed489fb3ce3b4cc8c9cf8a'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:00:17.172797 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-8c1ad391-f401-4beb-bc9c-994e4ab14ce1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5710749db2ed489fb3ce3b4cc8c9cf8a'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:00:17.173634 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-8c1ad391-f401-4beb-bc9c-994e4ab14ce1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5710749db2ed489fb3ce3b4cc8c9cf8a'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:00:17.186394 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c150fbfa-a788-4f9e-8074-1077f1410ee2 admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:00:17.187022 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1091/2171] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Sep 1 15:00:17 2022] DELETE /identity/v3/users/5710749db2ed489fb3ce3b4cc8c9cf8a => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 15:00:17.187793 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1ad391-f401-4beb-bc9c-994e4ab14ce1 admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:00:17.189084 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-8c1ad391-f401-4beb-bc9c-994e4ab14ce1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:00:17.253265 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4201576-30b5-4a3a-8340-ea2deb67f38f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:00:17.254044 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4201576-30b5-4a3a-8340-ea2deb67f38f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:00:17.254958 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4201576-30b5-4a3a-8340-ea2deb67f38f admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:00:17.255135 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4201576-30b5-4a3a-8340-ea2deb67f38f admin admin] PATH_INFO: `/v3/projects/3cde10dae67b460abbc07bc3936f8143` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:00:17.256226 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4201576-30b5-4a3a-8340-ea2deb67f38f admin admin] RBAC: Authorizing `identity:delete_project(project_id=3cde10dae67b460abbc07bc3936f8143)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:00:17.258183 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4201576-30b5-4a3a-8340-ea2deb67f38f admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:00:17.275092 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1ad391-f401-4beb-bc9c-994e4ab14ce1 admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:00:17.279966 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a1eb85-20d4-429b-aec8-8133b5945b03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:00:17.280736 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a1eb85-20d4-429b-aec8-8133b5945b03 admin admin] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:00:17.281021 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a1eb85-20d4-429b-aec8-8133b5945b03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:00:17.281292 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a1eb85-20d4-429b-aec8-8133b5945b03 admin admin] PATH_INFO: `/v3/users` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:00:17.282086 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a1eb85-20d4-429b-aec8-8133b5945b03 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:00:17.284129 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a1eb85-20d4-429b-aec8-8133b5945b03 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:00:17.311330 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1092/2172] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Sep 1 15:00:17 2022] POST /identity/v3/users => generated 287 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 15:00:17.312644 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250e4e51-9b26-4bcb-8050-486a85b2f7c9 None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:00:17.312935 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250e4e51-9b26-4bcb-8050-486a85b2f7c9 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:00:17.313227 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250e4e51-9b26-4bcb-8050-486a85b2f7c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:00:17.316001 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-a4201576-30b5-4a3a-8340-ea2deb67f38f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3cde10dae67b460abbc07bc3936f8143', 'request_id': 'req-a4201576-30b5-4a3a-8340-ea2deb67f38f'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:00:17.325009 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-a4201576-30b5-4a3a-8340-ea2deb67f38f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3cde10dae67b460abbc07bc3936f8143', 'request_id': 'req-a4201576-30b5-4a3a-8340-ea2deb67f38f'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:00:17.334367 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-250e4e51-9b26-4bcb-8050-486a85b2f7c9 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 15:00:17.335738 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-250e4e51-9b26-4bcb-8050-486a85b2f7c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:00:17.336443 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-a4201576-30b5-4a3a-8340-ea2deb67f38f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3cde10dae67b460abbc07bc3936f8143', 'request_id': 'req-a4201576-30b5-4a3a-8340-ea2deb67f38f'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:00:17.351074 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-a4201576-30b5-4a3a-8340-ea2deb67f38f admin admin] The token cache is being invalidate because project 3cde10dae67b460abbc07bc3936f8143 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81341) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 15:00:17.351385 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-a4201576-30b5-4a3a-8340-ea2deb67f38f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:00:17.352071 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-a4201576-30b5-4a3a-8340-ea2deb67f38f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:00:17.356835 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1081/2173] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Sep 1 15:00:17 2022] DELETE /identity/v3/projects/3cde10dae67b460abbc07bc3936f8143 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 15:00:17.356835 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4201576-30b5-4a3a-8340-ea2deb67f38f admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:00:17.356835 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-a4201576-30b5-4a3a-8340-ea2deb67f38f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:00:17.412643 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1093/2174] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 15:00:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 15:00:17.414153 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1055d57f-d4ef-4d0e-9f3a-358fdc755f46 None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:00:17.414527 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1055d57f-d4ef-4d0e-9f3a-358fdc755f46 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:00:17.414868 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1055d57f-d4ef-4d0e-9f3a-358fdc755f46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:00:17.441102 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-1055d57f-d4ef-4d0e-9f3a-358fdc755f46 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 15:00:17.442426 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-1055d57f-d4ef-4d0e-9f3a-358fdc755f46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:00:17.451262 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1094/2175] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 15:00:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 15:00:17.451262 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1055d57f-d4ef-4d0e-9f3a-358fdc755f46 None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:00:17.459311 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-1055d57f-d4ef-4d0e-9f3a-358fdc755f46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:00:17.467046 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4201576-30b5-4a3a-8340-ea2deb67f38f admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:00:17.472428 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2423e4c-44c9-4aee-8c4d-adcef839a50e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:00:17.473048 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2423e4c-44c9-4aee-8c4d-adcef839a50e admin admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:00:17.473330 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2423e4c-44c9-4aee-8c4d-adcef839a50e admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:00:17.473601 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2423e4c-44c9-4aee-8c4d-adcef839a50e admin admin] PATH_INFO: `/v3/roles` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:00:17.474867 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2423e4c-44c9-4aee-8c4d-adcef839a50e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:00:17.476500 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2423e4c-44c9-4aee-8c4d-adcef839a50e admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:00:17.486802 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1082/2176] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 1 15:00:17 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 15:00:17.487832 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9911648-a289-4977-b861-50e192608403 None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:00:17.488123 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9911648-a289-4977-b861-50e192608403 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:00:17.488393 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9911648-a289-4977-b861-50e192608403 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:00:17.507557 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-e9911648-a289-4977-b861-50e192608403 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 15:00:17.508771 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-e9911648-a289-4977-b861-50e192608403 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:00:17.519992 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1083/2177] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 15:00:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 15:00:17.519992 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9911648-a289-4977-b861-50e192608403 None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:00:17.520283 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1055d57f-d4ef-4d0e-9f3a-358fdc755f46 None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:00:17.523798 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 15:00:17.525333 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d4a4fe-df0b-4452-9076-c6e516959112 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2790149639aa418bbc4e4982e93eda44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:00:17.526269 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d4a4fe-df0b-4452-9076-c6e516959112 service neutron] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:00:17.526557 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d4a4fe-df0b-4452-9076-c6e516959112 service neutron] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:00:17.526822 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d4a4fe-df0b-4452-9076-c6e516959112 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:00:17.528280 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9911648-a289-4977-b861-50e192608403 None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:00:17.534961 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 15:00:17.535417 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06846f2-86ff-4441-8059-ed5b745e166f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:00:17.536134 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06846f2-86ff-4441-8059-ed5b745e166f admin admin] REQUEST_METHOD: `PUT` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:00:17.537265 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06846f2-86ff-4441-8059-ed5b745e166f admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:00:17.537705 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06846f2-86ff-4441-8059-ed5b745e166f admin admin] PATH_INFO: `/v3/projects/55990613114545109e3fe1eb571fcf4b/users/e241dfa36fde46ddb10d838f10c7e4ca/roles/2a2adb769e55476b82e5014897e0a9c0` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:00:17.538949 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d4a4fe-df0b-4452-9076-c6e516959112 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:00:17.540773 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d4a4fe-df0b-4452-9076-c6e516959112 service neutron] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:00:17.551773 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1095/2178] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 15:00:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 15:00:17.552950 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d4a4fe-df0b-4452-9076-c6e516959112 service neutron] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:00:17.553087 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06846f2-86ff-4441-8059-ed5b745e166f admin admin] RBAC: Authorizing `identity:create_grant(project_id=55990613114545109e3fe1eb571fcf4b, user_id=e241dfa36fde46ddb10d838f10c7e4ca, role_id=2a2adb769e55476b82e5014897e0a9c0)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:00:17.555775 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06846f2-86ff-4441-8059-ed5b745e166f admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:00:17.562728 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d4a4fe-df0b-4452-9076-c6e516959112 service neutron] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:00:17.570811 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1084/2179] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 1 15:00:17 2022] PUT /identity/v3/projects/55990613114545109e3fe1eb571fcf4b/users/e241dfa36fde46ddb10d838f10c7e4ca/roles/2a2adb769e55476b82e5014897e0a9c0 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 15:00:17.574852 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06846f2-86ff-4441-8059-ed5b745e166f admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:00:17.637408 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c304bf-93b4-4db0-9af2-5f79e871428e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:00:17.638174 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c304bf-93b4-4db0-9af2-5f79e871428e admin admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:00:17.638690 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c304bf-93b4-4db0-9af2-5f79e871428e admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:00:17.638791 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c304bf-93b4-4db0-9af2-5f79e871428e admin admin] PATH_INFO: `/v3/domains` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:00:17.639653 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c304bf-93b4-4db0-9af2-5f79e871428e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 15:00:17.640151 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c304bf-93b4-4db0-9af2-5f79e871428e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:00:17.641948 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c304bf-93b4-4db0-9af2-5f79e871428e admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:00:17.653299 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06846f2-86ff-4441-8059-ed5b745e166f admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:00:17.658269 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a4c3e2-be51-4cff-8099-c50663f4b0cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:00:17.659148 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a4c3e2-be51-4cff-8099-c50663f4b0cf admin admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:00:17.659539 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a4c3e2-be51-4cff-8099-c50663f4b0cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:00:17.659870 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a4c3e2-be51-4cff-8099-c50663f4b0cf admin admin] PATH_INFO: `/v3/roles` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:00:17.660661 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a4c3e2-be51-4cff-8099-c50663f4b0cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:00:17.662680 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a4c3e2-be51-4cff-8099-c50663f4b0cf admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:00:17.672257 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1096/2180] 10.222.0.33 () {58 vars in 1153 bytes} [Thu Sep 1 15:00:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 15:00:17.673248 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c304bf-93b4-4db0-9af2-5f79e871428e admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:00:17.677181 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1085/2181] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 1 15:00:17 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 15:00:17.684299 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a4c3e2-be51-4cff-8099-c50663f4b0cf admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:00:17.688163 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c304bf-93b4-4db0-9af2-5f79e871428e admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:00:17.694030 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d038aed9-fc03-4836-bac8-632e5ebd9251 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:00:17.694619 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d038aed9-fc03-4836-bac8-632e5ebd9251 admin admin] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:00:17.694897 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d038aed9-fc03-4836-bac8-632e5ebd9251 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:00:17.695168 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d038aed9-fc03-4836-bac8-632e5ebd9251 admin admin] PATH_INFO: `/v3/projects` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:00:17.695803 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d038aed9-fc03-4836-bac8-632e5ebd9251 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:00:17.697144 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a4c3e2-be51-4cff-8099-c50663f4b0cf admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:00:17.698270 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d038aed9-fc03-4836-bac8-632e5ebd9251 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:00:17.703109 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c4a4f4-c9b8-4a84-9310-35bc0fa8b3cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:00:17.704126 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c4a4f4-c9b8-4a84-9310-35bc0fa8b3cf admin admin] REQUEST_METHOD: `PUT` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:00:17.704545 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c4a4f4-c9b8-4a84-9310-35bc0fa8b3cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:00:17.704864 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c4a4f4-c9b8-4a84-9310-35bc0fa8b3cf admin admin] PATH_INFO: `/v3/projects/55990613114545109e3fe1eb571fcf4b/users/e241dfa36fde46ddb10d838f10c7e4ca/roles/2a2adb769e55476b82e5014897e0a9c0` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:00:17.707037 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c4a4f4-c9b8-4a84-9310-35bc0fa8b3cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=55990613114545109e3fe1eb571fcf4b, user_id=e241dfa36fde46ddb10d838f10c7e4ca, role_id=2a2adb769e55476b82e5014897e0a9c0)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:00:17.710449 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c4a4f4-c9b8-4a84-9310-35bc0fa8b3cf admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:00:17.727064 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1086/2182] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 1 15:00:17 2022] PUT /identity/v3/projects/55990613114545109e3fe1eb571fcf4b/users/e241dfa36fde46ddb10d838f10c7e4ca/roles/2a2adb769e55476b82e5014897e0a9c0 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 15:00:17.735735 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68d9182-8d11-4ebd-ad20-78ecc3fec2b2 None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:00:17.736109 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68d9182-8d11-4ebd-ad20-78ecc3fec2b2 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:00:17.736441 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68d9182-8d11-4ebd-ad20-78ecc3fec2b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:00:17.759982 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1097/2183] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Sep 1 15:00:17 2022] POST /identity/v3/projects => generated 371 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 15:00:17.763681 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d038aed9-fc03-4836-bac8-632e5ebd9251 admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:00:17.777945 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-c68d9182-8d11-4ebd-ad20-78ecc3fec2b2 None None] MFA Rules not processed for user `e241dfa36fde46ddb10d838f10c7e4ca`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 15:00:17.779497 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-c68d9182-8d11-4ebd-ad20-78ecc3fec2b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:00:17.836011 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d038aed9-fc03-4836-bac8-632e5ebd9251 admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:00:17.840827 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f14e729-7f75-4d2d-8588-edb479b7367e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:00:17.841491 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f14e729-7f75-4d2d-8588-edb479b7367e admin admin] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:00:17.841759 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f14e729-7f75-4d2d-8588-edb479b7367e admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:00:17.842072 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f14e729-7f75-4d2d-8588-edb479b7367e admin admin] PATH_INFO: `/v3/users` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:00:17.842843 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f14e729-7f75-4d2d-8588-edb479b7367e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:00:17.844843 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f14e729-7f75-4d2d-8588-edb479b7367e admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:00:17.868934 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1098/2184] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Sep 1 15:00:17 2022] POST /identity/v3/users => generated 286 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 15:00:17.874387 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f14e729-7f75-4d2d-8588-edb479b7367e admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:00:17.880223 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1087/2185] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 15:00:17 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 15:00:17.881020 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68d9182-8d11-4ebd-ad20-78ecc3fec2b2 None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:00:17.882483 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-c68d9182-8d11-4ebd-ad20-78ecc3fec2b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:00:17.890456 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f14e729-7f75-4d2d-8588-edb479b7367e admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:00:17.898897 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68d9182-8d11-4ebd-ad20-78ecc3fec2b2 None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:00:17.902213 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4366f6-a078-463c-a6ef-7ba9e32eca6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:00:17.902866 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4366f6-a078-463c-a6ef-7ba9e32eca6c admin admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:00:17.903140 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4366f6-a078-463c-a6ef-7ba9e32eca6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:00:17.904005 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4366f6-a078-463c-a6ef-7ba9e32eca6c admin admin] PATH_INFO: `/v3/roles` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:00:17.904660 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4366f6-a078-463c-a6ef-7ba9e32eca6c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:00:17.904766 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcf5d98-5d22-43f6-8d9b-1aca082020cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:00:17.905501 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcf5d98-5d22-43f6-8d9b-1aca082020cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:00:17.905812 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcf5d98-5d22-43f6-8d9b-1aca082020cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:00:17.906055 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcf5d98-5d22-43f6-8d9b-1aca082020cc admin admin] PATH_INFO: `/v3/projects/e70768e1f7f149578261a30ec295d59d` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:00:17.906617 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4366f6-a078-463c-a6ef-7ba9e32eca6c admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:00:17.907408 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fcf5d98-5d22-43f6-8d9b-1aca082020cc admin admin] RBAC: Authorizing `identity:delete_project(project_id=e70768e1f7f149578261a30ec295d59d)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:00:17.910212 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fcf5d98-5d22-43f6-8d9b-1aca082020cc admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:00:17.922018 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1099/2186] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 1 15:00:17 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 15:00:17.922018 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4366f6-a078-463c-a6ef-7ba9e32eca6c admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:00:17.922833 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-0c4366f6-a078-463c-a6ef-7ba9e32eca6c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:00:17.960655 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 15:00:17.964574 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-3fcf5d98-5d22-43f6-8d9b-1aca082020cc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e70768e1f7f149578261a30ec295d59d', 'request_id': 'req-3fcf5d98-5d22-43f6-8d9b-1aca082020cc'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:00:17.972046 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-3fcf5d98-5d22-43f6-8d9b-1aca082020cc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e70768e1f7f149578261a30ec295d59d', 'request_id': 'req-3fcf5d98-5d22-43f6-8d9b-1aca082020cc'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:00:17.980257 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-3fcf5d98-5d22-43f6-8d9b-1aca082020cc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e70768e1f7f149578261a30ec295d59d', 'request_id': 'req-3fcf5d98-5d22-43f6-8d9b-1aca082020cc'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:00:17.988176 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 15:00:17.991566 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-3fcf5d98-5d22-43f6-8d9b-1aca082020cc admin admin] The token cache is being invalidate because project e70768e1f7f149578261a30ec295d59d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81341) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 15:00:17.991953 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-3fcf5d98-5d22-43f6-8d9b-1aca082020cc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:00:17.992585 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-3fcf5d98-5d22-43f6-8d9b-1aca082020cc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:00:17.994315 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1088/2187] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Sep 1 15:00:17 2022] DELETE /identity/v3/projects/e70768e1f7f149578261a30ec295d59d => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 15:00:17.994571 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4366f6-a078-463c-a6ef-7ba9e32eca6c admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:00:17.995023 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcf5d98-5d22-43f6-8d9b-1aca082020cc admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:00:17.996302 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-3fcf5d98-5d22-43f6-8d9b-1aca082020cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:00:18.047526 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb305ce6-9da6-474a-834f-416e84330499 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3b81739e34e4e648f88b6d73fcb5e46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:00:18.047526 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb305ce6-9da6-474a-834f-416e84330499 service cinder] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:00:18.047853 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb305ce6-9da6-474a-834f-416e84330499 service cinder] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:00:18.047947 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb305ce6-9da6-474a-834f-416e84330499 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:00:18.049997 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-cb305ce6-9da6-474a-834f-416e84330499 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:00:18.064611 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcf5d98-5d22-43f6-8d9b-1aca082020cc admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:00:18.070278 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17cf41c-bd9d-4446-9f87-1bbbf24241a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:00:18.070943 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17cf41c-bd9d-4446-9f87-1bbbf24241a6 admin admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:00:18.071280 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17cf41c-bd9d-4446-9f87-1bbbf24241a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:00:18.071556 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17cf41c-bd9d-4446-9f87-1bbbf24241a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:00:18.072307 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17cf41c-bd9d-4446-9f87-1bbbf24241a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:00:18.074127 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17cf41c-bd9d-4446-9f87-1bbbf24241a6 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:00:18.085107 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1089/2188] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 1 15:00:17 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 15:00:18.086168 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028d4960-04fd-4533-aa0b-bd43ab4cecde None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:00:18.086465 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028d4960-04fd-4533-aa0b-bd43ab4cecde None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:00:18.086738 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028d4960-04fd-4533-aa0b-bd43ab4cecde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:00:18.106075 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-028d4960-04fd-4533-aa0b-bd43ab4cecde None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 15:00:18.107317 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-028d4960-04fd-4533-aa0b-bd43ab4cecde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:00:18.115061 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1090/2189] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 15:00:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 15:00:18.115789 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028d4960-04fd-4533-aa0b-bd43ab4cecde None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:00:18.116269 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb305ce6-9da6-474a-834f-416e84330499 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:00:18.118194 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb305ce6-9da6-474a-834f-416e84330499 service cinder] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:00:18.125711 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028d4960-04fd-4533-aa0b-bd43ab4cecde None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:00:18.127679 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1100/2190] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 15:00:17 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 15:00:18.130327 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa6ecf3-b186-4cf1-9a80-8ff959dced06 None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:00:18.130327 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa6ecf3-b186-4cf1-9a80-8ff959dced06 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:00:18.130327 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa6ecf3-b186-4cf1-9a80-8ff959dced06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:00:18.133375 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4bc32b-5899-4843-8d9c-d2fed551151d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:00:18.134197 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4bc32b-5899-4843-8d9c-d2fed551151d admin admin] REQUEST_METHOD: `PUT` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:00:18.134511 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4bc32b-5899-4843-8d9c-d2fed551151d admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:00:18.134844 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4bc32b-5899-4843-8d9c-d2fed551151d admin admin] PATH_INFO: `/v3/projects/d2633a2c5ec245979bf48998da59701e/users/e547206347b748039ff9395972188ab9/roles/5d1feef0e13d4874928bd86258c4e3b2` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:00:18.150890 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-3aa6ecf3-b186-4cf1-9a80-8ff959dced06 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 15:00:18.152104 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-3aa6ecf3-b186-4cf1-9a80-8ff959dced06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:00:18.152928 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4bc32b-5899-4843-8d9c-d2fed551151d admin admin] RBAC: Authorizing `identity:create_grant(project_id=d2633a2c5ec245979bf48998da59701e, user_id=e547206347b748039ff9395972188ab9, role_id=5d1feef0e13d4874928bd86258c4e3b2)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:00:18.156018 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4bc32b-5899-4843-8d9c-d2fed551151d admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:00:18.162433 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1101/2191] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 15:00:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 15:00:18.171260 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa6ecf3-b186-4cf1-9a80-8ff959dced06 None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:00:18.172018 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1091/2192] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 1 15:00:18 2022] PUT /identity/v3/projects/d2633a2c5ec245979bf48998da59701e/users/e547206347b748039ff9395972188ab9/roles/5d1feef0e13d4874928bd86258c4e3b2 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 15:00:18.181615 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa6ecf3-b186-4cf1-9a80-8ff959dced06 None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:00:18.182613 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5434260-f24a-4421-9d76-dc6f85f85257 None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:00:18.183044 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5434260-f24a-4421-9d76-dc6f85f85257 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:00:18.183313 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5434260-f24a-4421-9d76-dc6f85f85257 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:00:18.216149 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-d5434260-f24a-4421-9d76-dc6f85f85257 None None] MFA Rules not processed for user `e547206347b748039ff9395972188ab9`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 15:00:18.217844 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-d5434260-f24a-4421-9d76-dc6f85f85257 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:00:18.248183 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ff5e5c-e4f7-4bda-a623-0fb32bda8954 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:00:18.249009 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ff5e5c-e4f7-4bda-a623-0fb32bda8954 admin admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:00:18.249286 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ff5e5c-e4f7-4bda-a623-0fb32bda8954 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:00:18.249562 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ff5e5c-e4f7-4bda-a623-0fb32bda8954 admin admin] PATH_INFO: `/v3/domains` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:00:18.250209 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ff5e5c-e4f7-4bda-a623-0fb32bda8954 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 15:00:18.250585 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ff5e5c-e4f7-4bda-a623-0fb32bda8954 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:00:18.252221 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ff5e5c-e4f7-4bda-a623-0fb32bda8954 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:00:18.266600 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1102/2193] 10.222.0.33 () {58 vars in 1153 bytes} [Thu Sep 1 15:00:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 15:00:18.267585 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ff5e5c-e4f7-4bda-a623-0fb32bda8954 admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:00:18.269286 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-e5ff5e5c-e4f7-4bda-a623-0fb32bda8954 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:00:18.284631 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1092/2194] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 15:00:18 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 15:00:18.285523 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5434260-f24a-4421-9d76-dc6f85f85257 None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:00:18.299678 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5434260-f24a-4421-9d76-dc6f85f85257 None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:00:18.305704 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790d7702-8877-4ae9-85be-b168e6c14e24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:00:18.306340 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790d7702-8877-4ae9-85be-b168e6c14e24 admin admin] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:00:18.306625 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790d7702-8877-4ae9-85be-b168e6c14e24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:00:18.306900 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790d7702-8877-4ae9-85be-b168e6c14e24 admin admin] PATH_INFO: `/v3/projects` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:00:18.307561 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790d7702-8877-4ae9-85be-b168e6c14e24 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:00:18.309526 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790d7702-8877-4ae9-85be-b168e6c14e24 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:00:18.319340 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 15:00:18.334905 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ff5e5c-e4f7-4bda-a623-0fb32bda8954 admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:00:18.339378 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1093/2195] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Sep 1 15:00:18 2022] POST /identity/v3/projects => generated 385 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 15:00:18.340455 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09741be-86f3-4cdc-885d-81a1dd3e63c1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3b81739e34e4e648f88b6d73fcb5e46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:00:18.341113 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09741be-86f3-4cdc-885d-81a1dd3e63c1 service cinder] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:00:18.341420 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09741be-86f3-4cdc-885d-81a1dd3e63c1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:00:18.341615 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790d7702-8877-4ae9-85be-b168e6c14e24 admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:00:18.341718 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09741be-86f3-4cdc-885d-81a1dd3e63c1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:00:18.343284 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-790d7702-8877-4ae9-85be-b168e6c14e24 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:00:18.343483 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-a09741be-86f3-4cdc-885d-81a1dd3e63c1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:00:18.350156 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 15:00:18.402865 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790d7702-8877-4ae9-85be-b168e6c14e24 admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:00:18.406499 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1549249d-ae56-4204-a6d8-e6cc72f20dc4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3b81739e34e4e648f88b6d73fcb5e46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:00:18.407096 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1549249d-ae56-4204-a6d8-e6cc72f20dc4 service cinder] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:00:18.407374 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1549249d-ae56-4204-a6d8-e6cc72f20dc4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:00:18.407645 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1549249d-ae56-4204-a6d8-e6cc72f20dc4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:00:18.407999 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09741be-86f3-4cdc-885d-81a1dd3e63c1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:00:18.409457 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09741be-86f3-4cdc-885d-81a1dd3e63c1 service cinder] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:00:18.419626 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1103/2196] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 15:00:18 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 15:00:18.420569 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09741be-86f3-4cdc-885d-81a1dd3e63c1 service cinder] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:00:18.432619 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09741be-86f3-4cdc-885d-81a1dd3e63c1 service cinder] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:00:18.479657 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1549249d-ae56-4204-a6d8-e6cc72f20dc4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:00:18.481280 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1549249d-ae56-4204-a6d8-e6cc72f20dc4 service cinder] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:00:18.493163 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1094/2197] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 15:00:18 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 15:00:18.508472 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed32221-577a-4b85-8a6f-ffe121f4fa9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:00:18.509248 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed32221-577a-4b85-8a6f-ffe121f4fa9c admin admin] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:00:18.509630 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed32221-577a-4b85-8a6f-ffe121f4fa9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:00:18.510446 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed32221-577a-4b85-8a6f-ffe121f4fa9c admin admin] PATH_INFO: `/v3/users` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:00:18.511415 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed32221-577a-4b85-8a6f-ffe121f4fa9c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:00:18.513669 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed32221-577a-4b85-8a6f-ffe121f4fa9c admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:00:18.522287 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1549249d-ae56-4204-a6d8-e6cc72f20dc4 service cinder] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:00:18.535306 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1549249d-ae56-4204-a6d8-e6cc72f20dc4 service cinder] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:00:18.551141 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1104/2198] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Sep 1 15:00:18 2022] POST /identity/v3/users => generated 293 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 15:00:18.555999 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed32221-577a-4b85-8a6f-ffe121f4fa9c admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:00:18.562271 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28da97dc-c3ce-4915-807c-5a8bfd368a97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:00:18.563473 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28da97dc-c3ce-4915-807c-5a8bfd368a97 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:00:18.563473 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28da97dc-c3ce-4915-807c-5a8bfd368a97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:00:18.563626 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28da97dc-c3ce-4915-807c-5a8bfd368a97 admin admin] PATH_INFO: `/v3/users/e547206347b748039ff9395972188ab9` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:00:18.564711 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28da97dc-c3ce-4915-807c-5a8bfd368a97 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e547206347b748039ff9395972188ab9)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:00:18.567828 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28da97dc-c3ce-4915-807c-5a8bfd368a97 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:00:18.590049 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed32221-577a-4b85-8a6f-ffe121f4fa9c admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:00:18.599340 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4042df0f-3d67-4b91-9f7a-e7b9984627b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:00:18.603898 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4042df0f-3d67-4b91-9f7a-e7b9984627b3 admin admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:00:18.604230 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4042df0f-3d67-4b91-9f7a-e7b9984627b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:00:18.604507 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4042df0f-3d67-4b91-9f7a-e7b9984627b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:00:18.605256 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4042df0f-3d67-4b91-9f7a-e7b9984627b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:00:18.607929 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4042df0f-3d67-4b91-9f7a-e7b9984627b3 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:00:18.621729 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1105/2199] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 1 15:00:18 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 15:00:18.625695 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4042df0f-3d67-4b91-9f7a-e7b9984627b3 admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:00:18.640273 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4042df0f-3d67-4b91-9f7a-e7b9984627b3 admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:00:18.646676 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce11e747-8d7d-4dae-a7fd-a348cc45ce2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:00:18.647285 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce11e747-8d7d-4dae-a7fd-a348cc45ce2d admin admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:00:18.647578 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce11e747-8d7d-4dae-a7fd-a348cc45ce2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:00:18.647833 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce11e747-8d7d-4dae-a7fd-a348cc45ce2d admin admin] PATH_INFO: `/v3/roles` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:00:18.648482 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce11e747-8d7d-4dae-a7fd-a348cc45ce2d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:00:18.650692 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce11e747-8d7d-4dae-a7fd-a348cc45ce2d admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:00:18.659520 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-28da97dc-c3ce-4915-807c-5a8bfd368a97 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e547206347b748039ff9395972188ab9'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:00:18.669187 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1106/2200] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 1 15:00:18 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 15:00:18.673114 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce11e747-8d7d-4dae-a7fd-a348cc45ce2d admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:00:18.678762 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-28da97dc-c3ce-4915-807c-5a8bfd368a97 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e547206347b748039ff9395972188ab9'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:00:18.679816 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-28da97dc-c3ce-4915-807c-5a8bfd368a97 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e547206347b748039ff9395972188ab9'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:00:18.691529 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce11e747-8d7d-4dae-a7fd-a348cc45ce2d admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:00:18.696442 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-28da97dc-c3ce-4915-807c-5a8bfd368a97 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e547206347b748039ff9395972188ab9'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:00:18.697147 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-28da97dc-c3ce-4915-807c-5a8bfd368a97 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e547206347b748039ff9395972188ab9'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:00:18.697455 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2901f20f-e1ab-44a4-a9f6-b4e8e54c3cd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:00:18.698117 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2901f20f-e1ab-44a4-a9f6-b4e8e54c3cd5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:00:18.698590 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2901f20f-e1ab-44a4-a9f6-b4e8e54c3cd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:00:18.698867 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2901f20f-e1ab-44a4-a9f6-b4e8e54c3cd5 admin admin] PATH_INFO: `/v3/projects/5c12b0f429d04ac4b9b813377d409c4b/users/0c4c6a974e2f4ebbb3313584ed956e6a/roles/5d1feef0e13d4874928bd86258c4e3b2` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:00:18.709815 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1095/2201] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Sep 1 15:00:18 2022] DELETE /identity/v3/users/e547206347b748039ff9395972188ab9 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 15:00:18.714047 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1e8577-046a-4717-8f7e-925ef1d47ce3 None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:00:18.714397 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1e8577-046a-4717-8f7e-925ef1d47ce3 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:00:18.714676 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1e8577-046a-4717-8f7e-925ef1d47ce3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:00:18.717836 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2901f20f-e1ab-44a4-a9f6-b4e8e54c3cd5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c12b0f429d04ac4b9b813377d409c4b, user_id=0c4c6a974e2f4ebbb3313584ed956e6a, role_id=5d1feef0e13d4874928bd86258c4e3b2)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:00:18.720540 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2901f20f-e1ab-44a4-a9f6-b4e8e54c3cd5 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:00:18.733703 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-2f1e8577-046a-4717-8f7e-925ef1d47ce3 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 15:00:18.734418 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1107/2202] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 1 15:00:18 2022] PUT /identity/v3/projects/5c12b0f429d04ac4b9b813377d409c4b/users/0c4c6a974e2f4ebbb3313584ed956e6a/roles/5d1feef0e13d4874928bd86258c4e3b2 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 15:00:18.735107 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-2f1e8577-046a-4717-8f7e-925ef1d47ce3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:00:18.745414 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c95f7d4-4e28-45dd-a41e-fcecb98693bf None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:00:18.745704 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c95f7d4-4e28-45dd-a41e-fcecb98693bf None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:00:18.745980 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c95f7d4-4e28-45dd-a41e-fcecb98693bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:00:18.765398 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 15:00:18.775449 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-2c95f7d4-4e28-45dd-a41e-fcecb98693bf None None] MFA Rules not processed for user `0c4c6a974e2f4ebbb3313584ed956e6a`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 15:00:18.776858 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-2c95f7d4-4e28-45dd-a41e-fcecb98693bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:00:18.801638 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1096/2203] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 15:00:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 15:00:18.810619 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 15:00:18.814169 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1e8577-046a-4717-8f7e-925ef1d47ce3 None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:00:18.817173 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-2f1e8577-046a-4717-8f7e-925ef1d47ce3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:00:18.852914 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1108/2204] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 15:00:18 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 15:00:18.857984 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c95f7d4-4e28-45dd-a41e-fcecb98693bf None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:00:18.859457 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-2c95f7d4-4e28-45dd-a41e-fcecb98693bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:00:18.873362 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c95f7d4-4e28-45dd-a41e-fcecb98693bf None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:00:18.877017 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1e8577-046a-4717-8f7e-925ef1d47ce3 None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:00:18.878115 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa2c9fa-d405-4f98-954a-c9a4637213cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:00:18.878751 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa2c9fa-d405-4f98-954a-c9a4637213cd admin admin] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:00:18.879022 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa2c9fa-d405-4f98-954a-c9a4637213cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:00:18.879291 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa2c9fa-d405-4f98-954a-c9a4637213cd admin admin] PATH_INFO: `/v3/projects` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:00:18.879916 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa2c9fa-d405-4f98-954a-c9a4637213cd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:00:18.881391 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb2c7bd-585e-4f94-9421-59fccdad8923 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2790149639aa418bbc4e4982e93eda44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:00:18.881988 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb2c7bd-585e-4f94-9421-59fccdad8923 service neutron] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:00:18.882285 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb2c7bd-585e-4f94-9421-59fccdad8923 service neutron] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:00:18.882545 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa2c9fa-d405-4f98-954a-c9a4637213cd admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:00:18.882650 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb2c7bd-585e-4f94-9421-59fccdad8923 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:00:18.891640 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb2c7bd-585e-4f94-9421-59fccdad8923 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:00:18.893071 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb2c7bd-585e-4f94-9421-59fccdad8923 service neutron] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:00:18.902640 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1097/2205] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 15:00:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 15:00:18.913066 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1109/2206] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Sep 1 15:00:18 2022] POST /identity/v3/projects => generated 383 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 15:00:18.917337 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb2c7bd-585e-4f94-9421-59fccdad8923 service neutron] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:00:18.926869 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb2c7bd-585e-4f94-9421-59fccdad8923 service neutron] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:00:18.984779 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b809e4e8-aaf9-4171-af9a-cbe0b094651c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:00:18.985382 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b809e4e8-aaf9-4171-af9a-cbe0b094651c admin admin] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:00:18.985662 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b809e4e8-aaf9-4171-af9a-cbe0b094651c admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:00:18.985929 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b809e4e8-aaf9-4171-af9a-cbe0b094651c admin admin] PATH_INFO: `/v3/users` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:00:18.987016 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b809e4e8-aaf9-4171-af9a-cbe0b094651c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:00:18.988518 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b809e4e8-aaf9-4171-af9a-cbe0b094651c admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:00:19.013872 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1098/2207] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Sep 1 15:00:18 2022] POST /identity/v3/users => generated 292 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 15:00:19.015088 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa2c9fa-d405-4f98-954a-c9a4637213cd admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:00:19.025877 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa2c9fa-d405-4f98-954a-c9a4637213cd admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:00:19.031234 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23030df8-e5c1-4987-bcd9-f1e57b13d529 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:00:19.031836 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23030df8-e5c1-4987-bcd9-f1e57b13d529 admin admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:00:19.032111 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23030df8-e5c1-4987-bcd9-f1e57b13d529 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:00:19.032378 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23030df8-e5c1-4987-bcd9-f1e57b13d529 admin admin] PATH_INFO: `/v3/roles` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:00:19.033018 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23030df8-e5c1-4987-bcd9-f1e57b13d529 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:00:19.034636 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23030df8-e5c1-4987-bcd9-f1e57b13d529 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:00:19.046516 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1110/2208] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 1 15:00:19 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 15:00:19.050465 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b809e4e8-aaf9-4171-af9a-cbe0b094651c admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:00:19.060898 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b809e4e8-aaf9-4171-af9a-cbe0b094651c admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:00:19.065499 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e4cf12-71e7-43f8-a30a-6a0e14e9cb40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:00:19.066091 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e4cf12-71e7-43f8-a30a-6a0e14e9cb40 admin admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:00:19.066407 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e4cf12-71e7-43f8-a30a-6a0e14e9cb40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:00:19.066693 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e4cf12-71e7-43f8-a30a-6a0e14e9cb40 admin admin] PATH_INFO: `/v3/roles` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:00:19.067335 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e4cf12-71e7-43f8-a30a-6a0e14e9cb40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:00:19.069511 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e4cf12-71e7-43f8-a30a-6a0e14e9cb40 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:00:19.080722 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1099/2209] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 1 15:00:19 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 15:00:19.085667 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23030df8-e5c1-4987-bcd9-f1e57b13d529 admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:00:19.096540 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23030df8-e5c1-4987-bcd9-f1e57b13d529 admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:00:19.103488 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22920c1e-85d8-4fe2-b67d-78158190008e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:00:19.104162 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22920c1e-85d8-4fe2-b67d-78158190008e admin admin] REQUEST_METHOD: `PUT` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:00:19.104433 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22920c1e-85d8-4fe2-b67d-78158190008e admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:00:19.104702 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22920c1e-85d8-4fe2-b67d-78158190008e admin admin] PATH_INFO: `/v3/projects/65b91ea5c8974cf0bdd7aa18e52c7ca5/users/50accab656dc40b2808158f52351879d/roles/5d1feef0e13d4874928bd86258c4e3b2` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:00:19.120605 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22920c1e-85d8-4fe2-b67d-78158190008e admin admin] RBAC: Authorizing `identity:create_grant(project_id=65b91ea5c8974cf0bdd7aa18e52c7ca5, user_id=50accab656dc40b2808158f52351879d, role_id=5d1feef0e13d4874928bd86258c4e3b2)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:00:19.123306 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22920c1e-85d8-4fe2-b67d-78158190008e admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:00:19.138481 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1111/2210] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 1 15:00:19 2022] PUT /identity/v3/projects/65b91ea5c8974cf0bdd7aa18e52c7ca5/users/50accab656dc40b2808158f52351879d/roles/5d1feef0e13d4874928bd86258c4e3b2 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 15:00:19.149280 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb2a8f7-34aa-4c59-9e8c-f7199aa6edbe None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:00:19.149579 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb2a8f7-34aa-4c59-9e8c-f7199aa6edbe None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:00:19.149858 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb2a8f7-34aa-4c59-9e8c-f7199aa6edbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:00:19.180208 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-acb2a8f7-34aa-4c59-9e8c-f7199aa6edbe None None] MFA Rules not processed for user `50accab656dc40b2808158f52351879d`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 15:00:19.181539 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-acb2a8f7-34aa-4c59-9e8c-f7199aa6edbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:00:19.184066 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22920c1e-85d8-4fe2-b67d-78158190008e admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:00:19.186021 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-22920c1e-85d8-4fe2-b67d-78158190008e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:00:19.232108 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 15:00:19.256946 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22920c1e-85d8-4fe2-b67d-78158190008e admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:00:19.257658 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1100/2211] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 15:00:19 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 15:00:19.261753 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd3ea90-fb85-4af7-ab9a-815c8fdef333 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:00:19.262516 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd3ea90-fb85-4af7-ab9a-815c8fdef333 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:00:19.262684 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb2a8f7-34aa-4c59-9e8c-f7199aa6edbe None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:00:19.262919 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd3ea90-fb85-4af7-ab9a-815c8fdef333 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:00:19.263201 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd3ea90-fb85-4af7-ab9a-815c8fdef333 admin admin] PATH_INFO: `/v3/projects/d2633a2c5ec245979bf48998da59701e` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:00:19.264340 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd3ea90-fb85-4af7-ab9a-815c8fdef333 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d2633a2c5ec245979bf48998da59701e)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:00:19.266275 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd3ea90-fb85-4af7-ab9a-815c8fdef333 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:00:19.273666 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb2a8f7-34aa-4c59-9e8c-f7199aa6edbe None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:00:19.278141 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a04721-1c54-4f32-9440-5ce240b7f49e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:00:19.278748 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a04721-1c54-4f32-9440-5ce240b7f49e admin admin] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:00:19.279021 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a04721-1c54-4f32-9440-5ce240b7f49e admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:00:19.279289 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a04721-1c54-4f32-9440-5ce240b7f49e admin admin] PATH_INFO: `/v3/projects` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:00:19.279895 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a04721-1c54-4f32-9440-5ce240b7f49e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:00:19.281792 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a04721-1c54-4f32-9440-5ce240b7f49e admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:00:19.308897 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1101/2212] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Sep 1 15:00:19 2022] POST /identity/v3/projects => generated 383 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 15:00:19.313039 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a04721-1c54-4f32-9440-5ce240b7f49e admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:00:19.316896 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-ffd3ea90-fb85-4af7-ab9a-815c8fdef333 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd2633a2c5ec245979bf48998da59701e', 'request_id': 'req-ffd3ea90-fb85-4af7-ab9a-815c8fdef333'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:00:19.323944 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-ffd3ea90-fb85-4af7-ab9a-815c8fdef333 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd2633a2c5ec245979bf48998da59701e', 'request_id': 'req-ffd3ea90-fb85-4af7-ab9a-815c8fdef333'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:00:19.331535 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-ffd3ea90-fb85-4af7-ab9a-815c8fdef333 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd2633a2c5ec245979bf48998da59701e', 'request_id': 'req-ffd3ea90-fb85-4af7-ab9a-815c8fdef333'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:00:19.339827 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 15:00:19.343469 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-ffd3ea90-fb85-4af7-ab9a-815c8fdef333 admin admin] The token cache is being invalidate because project d2633a2c5ec245979bf48998da59701e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81342) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 15:00:19.344310 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-ffd3ea90-fb85-4af7-ab9a-815c8fdef333 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:00:19.345109 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-ffd3ea90-fb85-4af7-ab9a-815c8fdef333 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:00:19.347076 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1112/2213] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Sep 1 15:00:19 2022] DELETE /identity/v3/projects/d2633a2c5ec245979bf48998da59701e => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 15:00:19.387583 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a04721-1c54-4f32-9440-5ce240b7f49e admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:00:19.392602 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9513ff-0df4-4f68-bce3-0b40f32e66d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:00:19.393275 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9513ff-0df4-4f68-bce3-0b40f32e66d1 admin admin] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:00:19.393552 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9513ff-0df4-4f68-bce3-0b40f32e66d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:00:19.393824 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9513ff-0df4-4f68-bce3-0b40f32e66d1 admin admin] PATH_INFO: `/v3/users` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:00:19.394517 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9513ff-0df4-4f68-bce3-0b40f32e66d1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:00:19.397249 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9513ff-0df4-4f68-bce3-0b40f32e66d1 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:00:19.420755 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1102/2214] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Sep 1 15:00:19 2022] POST /identity/v3/users => generated 298 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 15:00:19.424858 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd3ea90-fb85-4af7-ab9a-815c8fdef333 admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:00:19.426507 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-ffd3ea90-fb85-4af7-ab9a-815c8fdef333 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:00:19.495763 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd3ea90-fb85-4af7-ab9a-815c8fdef333 admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:00:19.500516 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa12943a-b0ec-42d4-ae3b-85038bf1f6cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:00:19.501131 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa12943a-b0ec-42d4-ae3b-85038bf1f6cf admin admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:00:19.501407 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa12943a-b0ec-42d4-ae3b-85038bf1f6cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:00:19.501674 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa12943a-b0ec-42d4-ae3b-85038bf1f6cf admin admin] PATH_INFO: `/v3/roles` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:00:19.502361 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa12943a-b0ec-42d4-ae3b-85038bf1f6cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:00:19.503996 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa12943a-b0ec-42d4-ae3b-85038bf1f6cf admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:00:19.514741 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1113/2215] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 1 15:00:19 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 15:00:19.519001 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9513ff-0df4-4f68-bce3-0b40f32e66d1 admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:00:19.530168 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9513ff-0df4-4f68-bce3-0b40f32e66d1 admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:00:19.534767 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7555d0e-5421-4e9f-b7d8-63b797f96477 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:00:19.535421 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7555d0e-5421-4e9f-b7d8-63b797f96477 admin admin] REQUEST_METHOD: `PUT` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:00:19.535696 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7555d0e-5421-4e9f-b7d8-63b797f96477 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:00:19.535964 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7555d0e-5421-4e9f-b7d8-63b797f96477 admin admin] PATH_INFO: `/v3/projects/6d4b78654f044287b80db6ee64fc9722/users/e42c443b3fec4541ba2e95c31e32eac3/roles/2a2adb769e55476b82e5014897e0a9c0` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:00:19.551540 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7555d0e-5421-4e9f-b7d8-63b797f96477 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d4b78654f044287b80db6ee64fc9722, user_id=e42c443b3fec4541ba2e95c31e32eac3, role_id=2a2adb769e55476b82e5014897e0a9c0)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:00:19.554528 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7555d0e-5421-4e9f-b7d8-63b797f96477 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:00:19.567361 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1103/2216] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 1 15:00:19 2022] PUT /identity/v3/projects/6d4b78654f044287b80db6ee64fc9722/users/e42c443b3fec4541ba2e95c31e32eac3/roles/2a2adb769e55476b82e5014897e0a9c0 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 15:00:19.571373 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa12943a-b0ec-42d4-ae3b-85038bf1f6cf admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:00:19.581401 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa12943a-b0ec-42d4-ae3b-85038bf1f6cf admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:00:19.650474 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340eeda9-891e-4d70-b9e4-684b4d9b5f41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:00:19.656302 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340eeda9-891e-4d70-b9e4-684b4d9b5f41 admin admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:00:19.656302 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340eeda9-891e-4d70-b9e4-684b4d9b5f41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:00:19.656302 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340eeda9-891e-4d70-b9e4-684b4d9b5f41 admin admin] PATH_INFO: `/v3/roles` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:00:19.656302 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340eeda9-891e-4d70-b9e4-684b4d9b5f41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:00:19.656302 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340eeda9-891e-4d70-b9e4-684b4d9b5f41 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:00:19.664839 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1114/2217] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 1 15:00:19 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 15:00:19.669360 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7555d0e-5421-4e9f-b7d8-63b797f96477 admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:00:19.681212 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7555d0e-5421-4e9f-b7d8-63b797f96477 admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:00:19.686050 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5da2d13-29c5-42de-a634-1aa1ea5d80f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:00:19.686755 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5da2d13-29c5-42de-a634-1aa1ea5d80f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:00:19.687036 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5da2d13-29c5-42de-a634-1aa1ea5d80f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:00:19.687326 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5da2d13-29c5-42de-a634-1aa1ea5d80f0 admin admin] PATH_INFO: `/v3/projects/6d4b78654f044287b80db6ee64fc9722/users/e42c443b3fec4541ba2e95c31e32eac3/roles/2a2adb769e55476b82e5014897e0a9c0` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:00:19.689222 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5da2d13-29c5-42de-a634-1aa1ea5d80f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d4b78654f044287b80db6ee64fc9722, user_id=e42c443b3fec4541ba2e95c31e32eac3, role_id=2a2adb769e55476b82e5014897e0a9c0)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:00:19.692043 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5da2d13-29c5-42de-a634-1aa1ea5d80f0 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:00:19.709484 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1104/2218] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 1 15:00:19 2022] PUT /identity/v3/projects/6d4b78654f044287b80db6ee64fc9722/users/e42c443b3fec4541ba2e95c31e32eac3/roles/2a2adb769e55476b82e5014897e0a9c0 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 15:00:19.719128 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c822cb88-64f3-4f2d-95b9-919bf79080e9 None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:00:19.719538 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c822cb88-64f3-4f2d-95b9-919bf79080e9 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:00:19.719940 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c822cb88-64f3-4f2d-95b9-919bf79080e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:00:19.761912 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-c822cb88-64f3-4f2d-95b9-919bf79080e9 None None] MFA Rules not processed for user `e42c443b3fec4541ba2e95c31e32eac3`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 15:00:19.763170 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-c822cb88-64f3-4f2d-95b9-919bf79080e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:00:19.842421 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1115/2219] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 15:00:19 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 15:00:19.853734 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5da2d13-29c5-42de-a634-1aa1ea5d80f0 admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:00:19.855757 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-b5da2d13-29c5-42de-a634-1aa1ea5d80f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:00:19.920725 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5da2d13-29c5-42de-a634-1aa1ea5d80f0 admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:00:19.924356 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11779d20-3e11-4acb-907c-3de4c8517e27 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3b81739e34e4e648f88b6d73fcb5e46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:00:19.925129 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11779d20-3e11-4acb-907c-3de4c8517e27 service cinder] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:00:19.925406 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11779d20-3e11-4acb-907c-3de4c8517e27 service cinder] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:00:19.926202 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11779d20-3e11-4acb-907c-3de4c8517e27 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:00:19.928712 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-11779d20-3e11-4acb-907c-3de4c8517e27 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:00:19.957742 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 15:00:19.991524 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11779d20-3e11-4acb-907c-3de4c8517e27 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:00:19.992993 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11779d20-3e11-4acb-907c-3de4c8517e27 service cinder] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:00:20.002053 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1105/2220] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 15:00:19 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 15:00:21.253200 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c822cb88-64f3-4f2d-95b9-919bf79080e9 None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:00:21.254723 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-c822cb88-64f3-4f2d-95b9-919bf79080e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:00:21.275042 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 15:00:21.324805 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c822cb88-64f3-4f2d-95b9-919bf79080e9 None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:00:21.329421 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1ef271-e4bd-470b-9a40-94b71a145ea3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:00:21.330176 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1ef271-e4bd-470b-9a40-94b71a145ea3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:00:21.330390 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1ef271-e4bd-470b-9a40-94b71a145ea3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:00:21.330690 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1ef271-e4bd-470b-9a40-94b71a145ea3 admin admin] PATH_INFO: `/v3/users/67eea39c1f794dd89a51c15b1f46c593` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:00:21.331826 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1ef271-e4bd-470b-9a40-94b71a145ea3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=67eea39c1f794dd89a51c15b1f46c593)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:00:21.333796 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1ef271-e4bd-470b-9a40-94b71a145ea3 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:00:21.382751 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-cf1ef271-e4bd-470b-9a40-94b71a145ea3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '67eea39c1f794dd89a51c15b1f46c593'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:00:21.391918 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-cf1ef271-e4bd-470b-9a40-94b71a145ea3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '67eea39c1f794dd89a51c15b1f46c593'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:00:21.392582 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-cf1ef271-e4bd-470b-9a40-94b71a145ea3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '67eea39c1f794dd89a51c15b1f46c593'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:00:21.404985 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-cf1ef271-e4bd-470b-9a40-94b71a145ea3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '67eea39c1f794dd89a51c15b1f46c593'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:00:21.405658 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-cf1ef271-e4bd-470b-9a40-94b71a145ea3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '67eea39c1f794dd89a51c15b1f46c593'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:00:21.419337 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1116/2221] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Sep 1 15:00:21 2022] DELETE /identity/v3/users/67eea39c1f794dd89a51c15b1f46c593 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 15:00:21.423857 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae3e199-4502-4e8a-9f9e-29632c993962 None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:00:21.424664 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae3e199-4502-4e8a-9f9e-29632c993962 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:00:21.424943 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae3e199-4502-4e8a-9f9e-29632c993962 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:00:21.447472 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-cae3e199-4502-4e8a-9f9e-29632c993962 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 15:00:21.448712 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-cae3e199-4502-4e8a-9f9e-29632c993962 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:00:21.507152 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1ef271-e4bd-470b-9a40-94b71a145ea3 admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:00:21.508549 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-cf1ef271-e4bd-470b-9a40-94b71a145ea3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:00:21.511318 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1106/2222] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 15:00:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 15:00:21.520937 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae3e199-4502-4e8a-9f9e-29632c993962 None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:00:21.522282 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-cae3e199-4502-4e8a-9f9e-29632c993962 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:00:21.566589 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1ef271-e4bd-470b-9a40-94b71a145ea3 admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:00:21.570297 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce50027-2091-416a-b86c-7430d5d3a23d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3b81739e34e4e648f88b6d73fcb5e46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:00:21.571002 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce50027-2091-416a-b86c-7430d5d3a23d service cinder] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:00:21.571346 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce50027-2091-416a-b86c-7430d5d3a23d service cinder] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:00:21.571551 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce50027-2091-416a-b86c-7430d5d3a23d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:00:21.573239 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-9ce50027-2091-416a-b86c-7430d5d3a23d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:00:21.580088 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae3e199-4502-4e8a-9f9e-29632c993962 None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:00:21.583700 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d72fed-8cd7-4c2f-a30d-9256ae510cb0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2790149639aa418bbc4e4982e93eda44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:00:21.584296 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d72fed-8cd7-4c2f-a30d-9256ae510cb0 service neutron] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:00:21.584573 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d72fed-8cd7-4c2f-a30d-9256ae510cb0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:00:21.584842 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d72fed-8cd7-4c2f-a30d-9256ae510cb0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:00:21.593743 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d72fed-8cd7-4c2f-a30d-9256ae510cb0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:00:21.595213 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d72fed-8cd7-4c2f-a30d-9256ae510cb0 service neutron] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:00:21.605088 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1107/2223] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 15:00:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 15:00:21.622043 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 15:00:21.633724 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce50027-2091-416a-b86c-7430d5d3a23d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:00:21.635274 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce50027-2091-416a-b86c-7430d5d3a23d service cinder] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:00:21.644662 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1117/2224] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 15:00:21 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 15:00:21.893591 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d72fed-8cd7-4c2f-a30d-9256ae510cb0 service neutron] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:00:21.895506 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-05d72fed-8cd7-4c2f-a30d-9256ae510cb0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:00:21.912876 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d72fed-8cd7-4c2f-a30d-9256ae510cb0 service neutron] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:00:21.917720 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be16cc2-32bc-4735-a982-538b2cb098b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:00:21.918456 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be16cc2-32bc-4735-a982-538b2cb098b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:00:21.918667 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be16cc2-32bc-4735-a982-538b2cb098b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:00:21.918943 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be16cc2-32bc-4735-a982-538b2cb098b7 admin admin] PATH_INFO: `/v3/projects/728696de203440d38cd68ae50c3cbde5` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:00:21.920084 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be16cc2-32bc-4735-a982-538b2cb098b7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=728696de203440d38cd68ae50c3cbde5)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:00:21.922014 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be16cc2-32bc-4735-a982-538b2cb098b7 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:00:21.972929 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-3be16cc2-32bc-4735-a982-538b2cb098b7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '728696de203440d38cd68ae50c3cbde5', 'request_id': 'req-3be16cc2-32bc-4735-a982-538b2cb098b7'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:00:21.980045 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-3be16cc2-32bc-4735-a982-538b2cb098b7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '728696de203440d38cd68ae50c3cbde5', 'request_id': 'req-3be16cc2-32bc-4735-a982-538b2cb098b7'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:00:21.989520 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-3be16cc2-32bc-4735-a982-538b2cb098b7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '728696de203440d38cd68ae50c3cbde5', 'request_id': 'req-3be16cc2-32bc-4735-a982-538b2cb098b7'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:00:21.994770 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 15:00:22.002861 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-3be16cc2-32bc-4735-a982-538b2cb098b7 admin admin] The token cache is being invalidate because project 728696de203440d38cd68ae50c3cbde5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81341) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 15:00:22.003357 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-3be16cc2-32bc-4735-a982-538b2cb098b7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:00:22.004201 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-3be16cc2-32bc-4735-a982-538b2cb098b7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:00:22.005981 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1108/2225] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Sep 1 15:00:21 2022] DELETE /identity/v3/projects/728696de203440d38cd68ae50c3cbde5 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 15:00:22.010330 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce50027-2091-416a-b86c-7430d5d3a23d service cinder] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:00:22.022106 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce50027-2091-416a-b86c-7430d5d3a23d service cinder] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:00:22.084617 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd97ae3e-e2ed-41c1-83cc-7e7fdc4321cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:00:22.085261 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd97ae3e-e2ed-41c1-83cc-7e7fdc4321cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:00:22.085574 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd97ae3e-e2ed-41c1-83cc-7e7fdc4321cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:00:22.085814 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd97ae3e-e2ed-41c1-83cc-7e7fdc4321cf admin admin] PATH_INFO: `/v3/users/e241dfa36fde46ddb10d838f10c7e4ca` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:00:22.086993 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd97ae3e-e2ed-41c1-83cc-7e7fdc4321cf admin admin] RBAC: Authorizing `identity:delete_user(user_id=e241dfa36fde46ddb10d838f10c7e4ca)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:00:22.088957 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd97ae3e-e2ed-41c1-83cc-7e7fdc4321cf admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:00:22.147460 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-dd97ae3e-e2ed-41c1-83cc-7e7fdc4321cf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e241dfa36fde46ddb10d838f10c7e4ca'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:00:22.156802 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-dd97ae3e-e2ed-41c1-83cc-7e7fdc4321cf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e241dfa36fde46ddb10d838f10c7e4ca'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:00:22.158129 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-dd97ae3e-e2ed-41c1-83cc-7e7fdc4321cf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e241dfa36fde46ddb10d838f10c7e4ca'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:00:22.174184 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-dd97ae3e-e2ed-41c1-83cc-7e7fdc4321cf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e241dfa36fde46ddb10d838f10c7e4ca'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:00:22.174922 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-dd97ae3e-e2ed-41c1-83cc-7e7fdc4321cf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e241dfa36fde46ddb10d838f10c7e4ca'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:00:22.188293 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1118/2226] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Sep 1 15:00:22 2022] DELETE /identity/v3/users/e241dfa36fde46ddb10d838f10c7e4ca => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 15:00:22.224371 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be16cc2-32bc-4735-a982-538b2cb098b7 admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:00:22.225790 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-3be16cc2-32bc-4735-a982-538b2cb098b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:00:22.292665 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be16cc2-32bc-4735-a982-538b2cb098b7 admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:00:22.342660 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5796ab0-dfb0-4735-a2e4-52a6a3a8e329 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3b81739e34e4e648f88b6d73fcb5e46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:00:22.343267 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5796ab0-dfb0-4735-a2e4-52a6a3a8e329 service cinder] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:00:22.343545 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5796ab0-dfb0-4735-a2e4-52a6a3a8e329 service cinder] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:00:22.343819 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5796ab0-dfb0-4735-a2e4-52a6a3a8e329 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:00:22.345775 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-c5796ab0-dfb0-4735-a2e4-52a6a3a8e329 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:00:22.412288 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5796ab0-dfb0-4735-a2e4-52a6a3a8e329 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:00:22.413696 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5796ab0-dfb0-4735-a2e4-52a6a3a8e329 service cinder] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:00:22.423092 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1109/2227] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 15:00:22 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 15:00:22.482210 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd97ae3e-e2ed-41c1-83cc-7e7fdc4321cf admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:00:22.484675 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-dd97ae3e-e2ed-41c1-83cc-7e7fdc4321cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:00:22.560733 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd97ae3e-e2ed-41c1-83cc-7e7fdc4321cf admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:00:22.565373 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fac04f-8183-4438-8242-1d41df2f99ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:00:22.566002 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fac04f-8183-4438-8242-1d41df2f99ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:00:22.566332 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fac04f-8183-4438-8242-1d41df2f99ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:00:22.566586 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fac04f-8183-4438-8242-1d41df2f99ed admin admin] PATH_INFO: `/v3/projects/55990613114545109e3fe1eb571fcf4b` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:00:22.567730 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fac04f-8183-4438-8242-1d41df2f99ed admin admin] RBAC: Authorizing `identity:delete_project(project_id=55990613114545109e3fe1eb571fcf4b)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:00:22.569649 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fac04f-8183-4438-8242-1d41df2f99ed admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:00:22.580855 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 15:00:22.618879 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-21fac04f-8183-4438-8242-1d41df2f99ed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '55990613114545109e3fe1eb571fcf4b', 'request_id': 'req-21fac04f-8183-4438-8242-1d41df2f99ed'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:00:22.625131 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-21fac04f-8183-4438-8242-1d41df2f99ed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '55990613114545109e3fe1eb571fcf4b', 'request_id': 'req-21fac04f-8183-4438-8242-1d41df2f99ed'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:00:22.632659 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-21fac04f-8183-4438-8242-1d41df2f99ed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '55990613114545109e3fe1eb571fcf4b', 'request_id': 'req-21fac04f-8183-4438-8242-1d41df2f99ed'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:00:22.643110 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-21fac04f-8183-4438-8242-1d41df2f99ed admin admin] The token cache is being invalidate because project 55990613114545109e3fe1eb571fcf4b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81342) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 15:00:22.643413 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-21fac04f-8183-4438-8242-1d41df2f99ed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:00:22.644030 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-21fac04f-8183-4438-8242-1d41df2f99ed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:00:22.645587 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1119/2228] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Sep 1 15:00:22 2022] DELETE /identity/v3/projects/55990613114545109e3fe1eb571fcf4b => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 15:00:22.650953 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4964c0e2-88b4-447b-876b-325a586b93f9 None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:00:22.651436 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4964c0e2-88b4-447b-876b-325a586b93f9 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:00:22.651518 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4964c0e2-88b4-447b-876b-325a586b93f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:00:22.672036 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-4964c0e2-88b4-447b-876b-325a586b93f9 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 15:00:22.673360 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-4964c0e2-88b4-447b-876b-325a586b93f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:00:22.728384 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:DELETE=1,keystone:SELECT=99 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 15:00:22.737642 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1110/2229] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 15:00:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 15:00:22.745947 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac2635a-7862-4e53-b32d-38ee1aec536e None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:00:22.746229 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac2635a-7862-4e53-b32d-38ee1aec536e None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:00:22.746553 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac2635a-7862-4e53-b32d-38ee1aec536e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:00:22.767335 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-8ac2635a-7862-4e53-b32d-38ee1aec536e None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 15:00:22.768548 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-8ac2635a-7862-4e53-b32d-38ee1aec536e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:00:22.776542 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1120/2230] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 15:00:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 15:00:22.780744 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4964c0e2-88b4-447b-876b-325a586b93f9 None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:00:22.791282 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4964c0e2-88b4-447b-876b-325a586b93f9 None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:00:22.795895 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02eae205-cf58-42ee-bd9b-88a6d5e9d183 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:00:22.796550 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02eae205-cf58-42ee-bd9b-88a6d5e9d183 admin admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:00:22.796832 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02eae205-cf58-42ee-bd9b-88a6d5e9d183 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:00:22.797108 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02eae205-cf58-42ee-bd9b-88a6d5e9d183 admin admin] PATH_INFO: `/v3/domains` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:00:22.797757 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02eae205-cf58-42ee-bd9b-88a6d5e9d183 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81341) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 15:00:22.798059 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02eae205-cf58-42ee-bd9b-88a6d5e9d183 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:00:22.799736 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02eae205-cf58-42ee-bd9b-88a6d5e9d183 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:00:22.812359 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1111/2231] 10.222.0.33 () {58 vars in 1153 bytes} [Thu Sep 1 15:00:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 15:00:22.816591 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac2635a-7862-4e53-b32d-38ee1aec536e None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:00:22.826515 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac2635a-7862-4e53-b32d-38ee1aec536e None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:00:22.831132 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053a9712-9ef9-47a1-9bfa-d8bfd132ca39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:00:22.831717 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053a9712-9ef9-47a1-9bfa-d8bfd132ca39 admin admin] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:00:22.831993 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053a9712-9ef9-47a1-9bfa-d8bfd132ca39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:00:22.832262 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053a9712-9ef9-47a1-9bfa-d8bfd132ca39 admin admin] PATH_INFO: `/v3/projects` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:00:22.832886 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053a9712-9ef9-47a1-9bfa-d8bfd132ca39 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:00:22.834866 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053a9712-9ef9-47a1-9bfa-d8bfd132ca39 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:00:22.862188 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1121/2232] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Sep 1 15:00:22 2022] POST /identity/v3/projects => generated 377 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 15:00:22.867154 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02eae205-cf58-42ee-bd9b-88a6d5e9d183 admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:00:22.877460 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02eae205-cf58-42ee-bd9b-88a6d5e9d183 admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:00:22.934935 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9e8593-1e1e-45bb-85b9-260df400507c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:00:22.935550 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9e8593-1e1e-45bb-85b9-260df400507c admin admin] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:00:22.935799 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9e8593-1e1e-45bb-85b9-260df400507c admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:00:22.936111 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9e8593-1e1e-45bb-85b9-260df400507c admin admin] PATH_INFO: `/v3/users` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:00:22.936761 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9e8593-1e1e-45bb-85b9-260df400507c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:00:22.938915 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9e8593-1e1e-45bb-85b9-260df400507c admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:00:22.959128 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1112/2233] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Sep 1 15:00:22 2022] POST /identity/v3/users => generated 289 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 15:00:22.963219 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053a9712-9ef9-47a1-9bfa-d8bfd132ca39 admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:00:22.974635 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053a9712-9ef9-47a1-9bfa-d8bfd132ca39 admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:00:22.979146 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97be75bb-8982-43a3-bf8a-b5012fb26b6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:00:22.979733 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97be75bb-8982-43a3-bf8a-b5012fb26b6d admin admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:00:22.980009 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97be75bb-8982-43a3-bf8a-b5012fb26b6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:00:22.980276 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97be75bb-8982-43a3-bf8a-b5012fb26b6d admin admin] PATH_INFO: `/v3/roles` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:00:22.980919 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97be75bb-8982-43a3-bf8a-b5012fb26b6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:00:22.982592 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97be75bb-8982-43a3-bf8a-b5012fb26b6d admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:00:22.994056 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1122/2234] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 1 15:00:22 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 15:00:22.998027 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9e8593-1e1e-45bb-85b9-260df400507c admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:00:23.008598 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9e8593-1e1e-45bb-85b9-260df400507c admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:00:23.013343 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1f877b-f728-42c8-b010-42ec70f13c19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:00:23.013925 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1f877b-f728-42c8-b010-42ec70f13c19 admin admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:00:23.014202 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1f877b-f728-42c8-b010-42ec70f13c19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:00:23.014494 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1f877b-f728-42c8-b010-42ec70f13c19 admin admin] PATH_INFO: `/v3/roles` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:00:23.015129 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1f877b-f728-42c8-b010-42ec70f13c19 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:00:23.016758 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1f877b-f728-42c8-b010-42ec70f13c19 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:00:23.027543 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1113/2235] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 1 15:00:22 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 15:00:23.031825 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97be75bb-8982-43a3-bf8a-b5012fb26b6d admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:00:23.042325 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97be75bb-8982-43a3-bf8a-b5012fb26b6d admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:00:23.046983 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9661ac6b-c4b0-416a-90d0-8381b2a943f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:00:23.047627 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9661ac6b-c4b0-416a-90d0-8381b2a943f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:00:23.047909 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9661ac6b-c4b0-416a-90d0-8381b2a943f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:00:23.048185 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9661ac6b-c4b0-416a-90d0-8381b2a943f7 admin admin] PATH_INFO: `/v3/projects/2d2592fec05a428aa804075ad3aa8cd3/users/b68a36f8bcb04650bd950a0e6b31ec3c/roles/5d1feef0e13d4874928bd86258c4e3b2` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:00:23.064316 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9661ac6b-c4b0-416a-90d0-8381b2a943f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2d2592fec05a428aa804075ad3aa8cd3, user_id=b68a36f8bcb04650bd950a0e6b31ec3c, role_id=5d1feef0e13d4874928bd86258c4e3b2)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:00:23.067082 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9661ac6b-c4b0-416a-90d0-8381b2a943f7 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:00:23.079903 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1123/2236] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 1 15:00:23 2022] PUT /identity/v3/projects/2d2592fec05a428aa804075ad3aa8cd3/users/b68a36f8bcb04650bd950a0e6b31ec3c/roles/5d1feef0e13d4874928bd86258c4e3b2 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 15:00:23.091756 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3528bd-1bc5-49ba-b740-64a47a5d1f49 None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:00:23.092040 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3528bd-1bc5-49ba-b740-64a47a5d1f49 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:00:23.092315 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3528bd-1bc5-49ba-b740-64a47a5d1f49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:00:23.121250 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-1d3528bd-1bc5-49ba-b740-64a47a5d1f49 None None] MFA Rules not processed for user `b68a36f8bcb04650bd950a0e6b31ec3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 15:00:23.122497 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-1d3528bd-1bc5-49ba-b740-64a47a5d1f49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:00:23.185284 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1114/2237] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 15:00:23 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 15:00:23.189720 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9661ac6b-c4b0-416a-90d0-8381b2a943f7 admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:00:23.251014 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9661ac6b-c4b0-416a-90d0-8381b2a943f7 admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:00:23.255688 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5cee0d-d3cd-48ed-960d-33ef4ee9f2c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:00:23.256278 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5cee0d-d3cd-48ed-960d-33ef4ee9f2c9 admin admin] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:00:23.256556 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5cee0d-d3cd-48ed-960d-33ef4ee9f2c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:00:23.256827 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5cee0d-d3cd-48ed-960d-33ef4ee9f2c9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:00:23.257458 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5cee0d-d3cd-48ed-960d-33ef4ee9f2c9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:00:23.259400 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5cee0d-d3cd-48ed-960d-33ef4ee9f2c9 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:00:23.288265 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1124/2238] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Sep 1 15:00:23 2022] POST /identity/v3/projects => generated 377 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 15:00:23.292240 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3528bd-1bc5-49ba-b740-64a47a5d1f49 None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:00:23.308343 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3528bd-1bc5-49ba-b740-64a47a5d1f49 None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:00:23.361076 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 15:00:23.369814 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e571a64-6533-4fe0-95eb-bb70fc53cbd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:00:23.370461 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e571a64-6533-4fe0-95eb-bb70fc53cbd1 admin admin] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:00:23.370740 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e571a64-6533-4fe0-95eb-bb70fc53cbd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:00:23.371013 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e571a64-6533-4fe0-95eb-bb70fc53cbd1 admin admin] PATH_INFO: `/v3/users` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:00:23.371652 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e571a64-6533-4fe0-95eb-bb70fc53cbd1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:00:23.373591 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e571a64-6533-4fe0-95eb-bb70fc53cbd1 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:00:23.399566 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1115/2239] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Sep 1 15:00:23 2022] POST /identity/v3/users => generated 295 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 15:00:23.400903 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5cee0d-d3cd-48ed-960d-33ef4ee9f2c9 admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:00:23.412250 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5cee0d-d3cd-48ed-960d-33ef4ee9f2c9 admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:00:23.416968 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c08c311-be9d-41b0-a692-c53a075bf211 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:00:23.417567 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c08c311-be9d-41b0-a692-c53a075bf211 admin admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:00:23.417850 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c08c311-be9d-41b0-a692-c53a075bf211 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:00:23.418137 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c08c311-be9d-41b0-a692-c53a075bf211 admin admin] PATH_INFO: `/v3/roles` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:00:23.418805 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c08c311-be9d-41b0-a692-c53a075bf211 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:00:23.420466 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c08c311-be9d-41b0-a692-c53a075bf211 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:00:23.436993 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1125/2240] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 1 15:00:23 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 15:00:23.441078 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e571a64-6533-4fe0-95eb-bb70fc53cbd1 admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:00:23.451077 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e571a64-6533-4fe0-95eb-bb70fc53cbd1 admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:00:23.455664 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ceb969-80f0-4b1c-86c9-657d29584617 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:00:23.456300 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ceb969-80f0-4b1c-86c9-657d29584617 admin admin] REQUEST_METHOD: `PUT` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:00:23.456578 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ceb969-80f0-4b1c-86c9-657d29584617 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:00:23.456847 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ceb969-80f0-4b1c-86c9-657d29584617 admin admin] PATH_INFO: `/v3/projects/ed703d1205ae4e3d9e5a29409b34329b/users/7cb20e8a8dd345aebd335d858bb77f9a/roles/2a2adb769e55476b82e5014897e0a9c0` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:00:23.471559 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ceb969-80f0-4b1c-86c9-657d29584617 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed703d1205ae4e3d9e5a29409b34329b, user_id=7cb20e8a8dd345aebd335d858bb77f9a, role_id=2a2adb769e55476b82e5014897e0a9c0)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:00:23.474891 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ceb969-80f0-4b1c-86c9-657d29584617 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:00:23.489830 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1116/2241] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 1 15:00:23 2022] PUT /identity/v3/projects/ed703d1205ae4e3d9e5a29409b34329b/users/7cb20e8a8dd345aebd335d858bb77f9a/roles/2a2adb769e55476b82e5014897e0a9c0 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 15:00:23.493708 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c08c311-be9d-41b0-a692-c53a075bf211 admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:00:23.503595 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c08c311-be9d-41b0-a692-c53a075bf211 admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:00:23.511315 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 15:00:23.562001 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef07801-2370-44c1-b392-8e1a999ef10c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:00:23.562810 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef07801-2370-44c1-b392-8e1a999ef10c admin admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:00:23.563087 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef07801-2370-44c1-b392-8e1a999ef10c admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:00:23.563353 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef07801-2370-44c1-b392-8e1a999ef10c admin admin] PATH_INFO: `/v3/roles` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:00:23.563994 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef07801-2370-44c1-b392-8e1a999ef10c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:00:23.565582 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef07801-2370-44c1-b392-8e1a999ef10c admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:00:23.576533 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1126/2242] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 1 15:00:23 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 15:00:23.580618 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ceb969-80f0-4b1c-86c9-657d29584617 admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:00:23.591507 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ceb969-80f0-4b1c-86c9-657d29584617 admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:00:23.596075 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbafbd5-ef00-4fb6-877a-45a965447aad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:00:23.596714 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbafbd5-ef00-4fb6-877a-45a965447aad admin admin] REQUEST_METHOD: `PUT` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:00:23.596994 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbafbd5-ef00-4fb6-877a-45a965447aad admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:00:23.597268 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbafbd5-ef00-4fb6-877a-45a965447aad admin admin] PATH_INFO: `/v3/projects/ed703d1205ae4e3d9e5a29409b34329b/users/7cb20e8a8dd345aebd335d858bb77f9a/roles/2a2adb769e55476b82e5014897e0a9c0` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:00:23.599228 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbafbd5-ef00-4fb6-877a-45a965447aad admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed703d1205ae4e3d9e5a29409b34329b, user_id=7cb20e8a8dd345aebd335d858bb77f9a, role_id=2a2adb769e55476b82e5014897e0a9c0)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:00:23.601847 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbafbd5-ef00-4fb6-877a-45a965447aad admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:00:23.611908 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1117/2243] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 1 15:00:23 2022] PUT /identity/v3/projects/ed703d1205ae4e3d9e5a29409b34329b/users/7cb20e8a8dd345aebd335d858bb77f9a/roles/2a2adb769e55476b82e5014897e0a9c0 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 15:00:23.621866 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09577334-e24a-4ed9-a9fc-984f8c4920b2 None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:00:23.622152 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09577334-e24a-4ed9-a9fc-984f8c4920b2 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:00:23.622457 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09577334-e24a-4ed9-a9fc-984f8c4920b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:00:23.650761 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-09577334-e24a-4ed9-a9fc-984f8c4920b2 None None] MFA Rules not processed for user `7cb20e8a8dd345aebd335d858bb77f9a`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 15:00:23.651989 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-09577334-e24a-4ed9-a9fc-984f8c4920b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:00:23.730387 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1127/2244] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 15:00:23 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 15:00:23.741071 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbafbd5-ef00-4fb6-877a-45a965447aad admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:00:23.742865 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-fcbafbd5-ef00-4fb6-877a-45a965447aad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:00:23.811802 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbafbd5-ef00-4fb6-877a-45a965447aad admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:00:23.818123 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bae6bc-6bd0-4f68-a0e0-b2b83194259e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3b81739e34e4e648f88b6d73fcb5e46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:00:23.818952 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bae6bc-6bd0-4f68-a0e0-b2b83194259e service cinder] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:00:23.819296 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bae6bc-6bd0-4f68-a0e0-b2b83194259e service cinder] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:00:23.819665 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bae6bc-6bd0-4f68-a0e0-b2b83194259e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:00:23.896399 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bae6bc-6bd0-4f68-a0e0-b2b83194259e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:00:23.897877 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bae6bc-6bd0-4f68-a0e0-b2b83194259e service cinder] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:00:23.907779 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1118/2245] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 15:00:23 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 15:00:31.634631 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09577334-e24a-4ed9-a9fc-984f8c4920b2 None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:00:31.636041 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-09577334-e24a-4ed9-a9fc-984f8c4920b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:00:31.648222 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09577334-e24a-4ed9-a9fc-984f8c4920b2 None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:00:31.651981 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ef5f7c-0822-4322-a39c-9095ba47b608 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3b81739e34e4e648f88b6d73fcb5e46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:00:31.652588 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ef5f7c-0822-4322-a39c-9095ba47b608 service cinder] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:00:31.652861 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ef5f7c-0822-4322-a39c-9095ba47b608 service cinder] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:00:31.653123 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ef5f7c-0822-4322-a39c-9095ba47b608 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:00:31.664255 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ef5f7c-0822-4322-a39c-9095ba47b608 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:00:31.665328 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ef5f7c-0822-4322-a39c-9095ba47b608 service cinder] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:00:31.675560 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1128/2246] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 15:00:31 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 15:00:32.628836 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bae6bc-6bd0-4f68-a0e0-b2b83194259e service cinder] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:00:32.630776 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-35bae6bc-6bd0-4f68-a0e0-b2b83194259e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:00:32.697246 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bae6bc-6bd0-4f68-a0e0-b2b83194259e service cinder] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:00:32.701739 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb8beea-c7d1-4260-a98f-cfd76acbe1d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:00:32.702398 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb8beea-c7d1-4260-a98f-cfd76acbe1d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:00:32.702678 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb8beea-c7d1-4260-a98f-cfd76acbe1d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:00:32.702945 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb8beea-c7d1-4260-a98f-cfd76acbe1d1 admin admin] PATH_INFO: `/v3/users/0c4c6a974e2f4ebbb3313584ed956e6a` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:00:32.704043 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb8beea-c7d1-4260-a98f-cfd76acbe1d1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0c4c6a974e2f4ebbb3313584ed956e6a)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:00:32.705989 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb8beea-c7d1-4260-a98f-cfd76acbe1d1 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:00:32.708240 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 15:00:32.753507 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-ccb8beea-c7d1-4260-a98f-cfd76acbe1d1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0c4c6a974e2f4ebbb3313584ed956e6a'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:00:32.761079 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-ccb8beea-c7d1-4260-a98f-cfd76acbe1d1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0c4c6a974e2f4ebbb3313584ed956e6a'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:00:32.761833 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-ccb8beea-c7d1-4260-a98f-cfd76acbe1d1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0c4c6a974e2f4ebbb3313584ed956e6a'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:00:32.774863 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-ccb8beea-c7d1-4260-a98f-cfd76acbe1d1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0c4c6a974e2f4ebbb3313584ed956e6a'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:00:32.775559 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-ccb8beea-c7d1-4260-a98f-cfd76acbe1d1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0c4c6a974e2f4ebbb3313584ed956e6a'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:00:32.789582 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1119/2247] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Sep 1 15:00:32 2022] DELETE /identity/v3/users/0c4c6a974e2f4ebbb3313584ed956e6a => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 15:00:32.794343 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32517163-1228-4e91-9320-c63d0e60d952 None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:00:32.794831 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32517163-1228-4e91-9320-c63d0e60d952 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:00:32.795017 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32517163-1228-4e91-9320-c63d0e60d952 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:00:32.814730 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-32517163-1228-4e91-9320-c63d0e60d952 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 15:00:32.816021 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-32517163-1228-4e91-9320-c63d0e60d952 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:00:32.875498 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1129/2248] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 15:00:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 15:00:32.885559 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb8beea-c7d1-4260-a98f-cfd76acbe1d1 admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:00:32.887142 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-ccb8beea-c7d1-4260-a98f-cfd76acbe1d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:00:32.945679 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb8beea-c7d1-4260-a98f-cfd76acbe1d1 admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:00:32.949279 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd73ba84-dd28-4b70-b930-4ae4ecc76240 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2790149639aa418bbc4e4982e93eda44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:00:32.949869 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd73ba84-dd28-4b70-b930-4ae4ecc76240 service neutron] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:00:32.950143 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd73ba84-dd28-4b70-b930-4ae4ecc76240 service neutron] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:00:32.950439 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd73ba84-dd28-4b70-b930-4ae4ecc76240 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:00:32.959818 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd73ba84-dd28-4b70-b930-4ae4ecc76240 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:00:32.961257 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd73ba84-dd28-4b70-b930-4ae4ecc76240 service neutron] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:00:32.971008 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1120/2249] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 15:00:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 15:00:33.265976 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32517163-1228-4e91-9320-c63d0e60d952 None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:00:33.267407 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-32517163-1228-4e91-9320-c63d0e60d952 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:00:33.281038 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32517163-1228-4e91-9320-c63d0e60d952 None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:00:33.285807 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0791bff4-3fa0-4535-b58d-6175a7364d47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:00:33.287182 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0791bff4-3fa0-4535-b58d-6175a7364d47 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:00:33.287182 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0791bff4-3fa0-4535-b58d-6175a7364d47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:00:33.287182 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0791bff4-3fa0-4535-b58d-6175a7364d47 admin admin] PATH_INFO: `/v3/projects/5c12b0f429d04ac4b9b813377d409c4b` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:00:33.289054 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0791bff4-3fa0-4535-b58d-6175a7364d47 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5c12b0f429d04ac4b9b813377d409c4b)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:00:33.291212 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0791bff4-3fa0-4535-b58d-6175a7364d47 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:00:33.311697 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 15:00:33.349784 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-0791bff4-3fa0-4535-b58d-6175a7364d47 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5c12b0f429d04ac4b9b813377d409c4b', 'request_id': 'req-0791bff4-3fa0-4535-b58d-6175a7364d47'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:00:33.362183 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-0791bff4-3fa0-4535-b58d-6175a7364d47 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5c12b0f429d04ac4b9b813377d409c4b', 'request_id': 'req-0791bff4-3fa0-4535-b58d-6175a7364d47'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:00:33.369681 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-0791bff4-3fa0-4535-b58d-6175a7364d47 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5c12b0f429d04ac4b9b813377d409c4b', 'request_id': 'req-0791bff4-3fa0-4535-b58d-6175a7364d47'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:00:33.380413 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-0791bff4-3fa0-4535-b58d-6175a7364d47 admin admin] The token cache is being invalidate because project 5c12b0f429d04ac4b9b813377d409c4b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81342) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 15:00:33.380699 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-0791bff4-3fa0-4535-b58d-6175a7364d47 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:00:33.381887 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-0791bff4-3fa0-4535-b58d-6175a7364d47 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:00:33.383673 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1130/2250] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Sep 1 15:00:33 2022] DELETE /identity/v3/projects/5c12b0f429d04ac4b9b813377d409c4b => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 15:00:33.387619 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd73ba84-dd28-4b70-b930-4ae4ecc76240 service neutron] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:00:33.398027 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd73ba84-dd28-4b70-b930-4ae4ecc76240 service neutron] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:00:33.452859 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0118c3c-386c-4788-b6b7-31860846709f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:00:33.453474 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0118c3c-386c-4788-b6b7-31860846709f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:00:33.453753 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0118c3c-386c-4788-b6b7-31860846709f admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:00:33.454027 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0118c3c-386c-4788-b6b7-31860846709f admin admin] PATH_INFO: `/v3/users/50accab656dc40b2808158f52351879d` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:00:33.455173 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0118c3c-386c-4788-b6b7-31860846709f admin admin] RBAC: Authorizing `identity:delete_user(user_id=50accab656dc40b2808158f52351879d)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:00:33.457102 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0118c3c-386c-4788-b6b7-31860846709f admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:00:33.486885 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 15:00:33.508063 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-d0118c3c-386c-4788-b6b7-31860846709f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '50accab656dc40b2808158f52351879d'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:00:33.515674 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-d0118c3c-386c-4788-b6b7-31860846709f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '50accab656dc40b2808158f52351879d'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:00:33.516355 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-d0118c3c-386c-4788-b6b7-31860846709f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '50accab656dc40b2808158f52351879d'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:00:33.528392 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-d0118c3c-386c-4788-b6b7-31860846709f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '50accab656dc40b2808158f52351879d'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:00:33.529015 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-d0118c3c-386c-4788-b6b7-31860846709f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '50accab656dc40b2808158f52351879d'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:00:33.541103 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1121/2251] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Sep 1 15:00:33 2022] DELETE /identity/v3/users/50accab656dc40b2808158f52351879d => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 15:00:33.830653 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0791bff4-3fa0-4535-b58d-6175a7364d47 admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:00:33.832135 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-0791bff4-3fa0-4535-b58d-6175a7364d47 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:00:33.845242 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0791bff4-3fa0-4535-b58d-6175a7364d47 admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:00:33.909676 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2369497f-195b-4316-963e-fb83e88b4709 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:00:33.910336 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2369497f-195b-4316-963e-fb83e88b4709 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:00:33.910614 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2369497f-195b-4316-963e-fb83e88b4709 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:00:33.910913 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2369497f-195b-4316-963e-fb83e88b4709 admin admin] PATH_INFO: `/v3/projects/65b91ea5c8974cf0bdd7aa18e52c7ca5` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:00:33.912003 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2369497f-195b-4316-963e-fb83e88b4709 admin admin] RBAC: Authorizing `identity:delete_project(project_id=65b91ea5c8974cf0bdd7aa18e52c7ca5)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:00:33.913983 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2369497f-195b-4316-963e-fb83e88b4709 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:00:33.961387 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-2369497f-195b-4316-963e-fb83e88b4709 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '65b91ea5c8974cf0bdd7aa18e52c7ca5', 'request_id': 'req-2369497f-195b-4316-963e-fb83e88b4709'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:00:33.967638 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-2369497f-195b-4316-963e-fb83e88b4709 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '65b91ea5c8974cf0bdd7aa18e52c7ca5', 'request_id': 'req-2369497f-195b-4316-963e-fb83e88b4709'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:00:33.975073 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-2369497f-195b-4316-963e-fb83e88b4709 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '65b91ea5c8974cf0bdd7aa18e52c7ca5', 'request_id': 'req-2369497f-195b-4316-963e-fb83e88b4709'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:00:33.985296 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-2369497f-195b-4316-963e-fb83e88b4709 admin admin] The token cache is being invalidate because project 65b91ea5c8974cf0bdd7aa18e52c7ca5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81342) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 15:00:33.985573 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-2369497f-195b-4316-963e-fb83e88b4709 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:00:33.986206 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-2369497f-195b-4316-963e-fb83e88b4709 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:00:33.987713 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1131/2252] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Sep 1 15:00:33 2022] DELETE /identity/v3/projects/65b91ea5c8974cf0bdd7aa18e52c7ca5 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 15:00:33.992109 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0118c3c-386c-4788-b6b7-31860846709f admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:00:33.993520 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-d0118c3c-386c-4788-b6b7-31860846709f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:00:34.006520 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0118c3c-386c-4788-b6b7-31860846709f admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:00:34.066069 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e288778-da40-4dcd-9577-315be1ee9e67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:00:34.066848 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e288778-da40-4dcd-9577-315be1ee9e67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:00:34.067189 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e288778-da40-4dcd-9577-315be1ee9e67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:00:34.067522 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e288778-da40-4dcd-9577-315be1ee9e67 admin admin] PATH_INFO: `/v3/users/e42c443b3fec4541ba2e95c31e32eac3` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:00:34.068805 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e288778-da40-4dcd-9577-315be1ee9e67 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e42c443b3fec4541ba2e95c31e32eac3)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:00:34.070830 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e288778-da40-4dcd-9577-315be1ee9e67 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:00:34.127953 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-1e288778-da40-4dcd-9577-315be1ee9e67 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e42c443b3fec4541ba2e95c31e32eac3'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:00:34.135224 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-1e288778-da40-4dcd-9577-315be1ee9e67 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e42c443b3fec4541ba2e95c31e32eac3'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:00:34.135817 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-1e288778-da40-4dcd-9577-315be1ee9e67 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e42c443b3fec4541ba2e95c31e32eac3'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:00:34.147630 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-1e288778-da40-4dcd-9577-315be1ee9e67 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e42c443b3fec4541ba2e95c31e32eac3'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:00:34.148255 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-1e288778-da40-4dcd-9577-315be1ee9e67 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e42c443b3fec4541ba2e95c31e32eac3'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:00:34.160169 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1122/2253] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Sep 1 15:00:33 2022] DELETE /identity/v3/users/e42c443b3fec4541ba2e95c31e32eac3 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 15:00:34.446664 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2369497f-195b-4316-963e-fb83e88b4709 admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:00:34.448090 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-2369497f-195b-4316-963e-fb83e88b4709 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:00:34.466811 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2369497f-195b-4316-963e-fb83e88b4709 admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:00:34.500239 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 15:00:34.529649 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ad42c2-3c6f-4366-955e-b182c41cbf0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:00:34.530311 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ad42c2-3c6f-4366-955e-b182c41cbf0e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:00:34.530595 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ad42c2-3c6f-4366-955e-b182c41cbf0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:00:34.530867 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ad42c2-3c6f-4366-955e-b182c41cbf0e admin admin] PATH_INFO: `/v3/projects/6d4b78654f044287b80db6ee64fc9722` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:00:34.531988 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ad42c2-3c6f-4366-955e-b182c41cbf0e admin admin] RBAC: Authorizing `identity:delete_project(project_id=6d4b78654f044287b80db6ee64fc9722)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:00:34.533916 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ad42c2-3c6f-4366-955e-b182c41cbf0e admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:00:34.580021 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-64ad42c2-3c6f-4366-955e-b182c41cbf0e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6d4b78654f044287b80db6ee64fc9722', 'request_id': 'req-64ad42c2-3c6f-4366-955e-b182c41cbf0e'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:00:34.586614 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-64ad42c2-3c6f-4366-955e-b182c41cbf0e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6d4b78654f044287b80db6ee64fc9722', 'request_id': 'req-64ad42c2-3c6f-4366-955e-b182c41cbf0e'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:00:34.594161 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-64ad42c2-3c6f-4366-955e-b182c41cbf0e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6d4b78654f044287b80db6ee64fc9722', 'request_id': 'req-64ad42c2-3c6f-4366-955e-b182c41cbf0e'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:00:34.605341 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-64ad42c2-3c6f-4366-955e-b182c41cbf0e admin admin] The token cache is being invalidate because project 6d4b78654f044287b80db6ee64fc9722 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81342) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 15:00:34.605626 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-64ad42c2-3c6f-4366-955e-b182c41cbf0e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:00:34.606261 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-64ad42c2-3c6f-4366-955e-b182c41cbf0e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:00:34.607735 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1132/2254] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Sep 1 15:00:34 2022] DELETE /identity/v3/projects/6d4b78654f044287b80db6ee64fc9722 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 15:00:34.612615 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305c50b8-03d5-4aab-a3d6-42d10eda5244 None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:00:34.612902 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305c50b8-03d5-4aab-a3d6-42d10eda5244 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:00:34.613178 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305c50b8-03d5-4aab-a3d6-42d10eda5244 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:00:34.632809 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-305c50b8-03d5-4aab-a3d6-42d10eda5244 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 15:00:34.634005 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-305c50b8-03d5-4aab-a3d6-42d10eda5244 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:00:34.650763 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:DELETE=16,keystone:SELECT=82,keystone:INSERT=2 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 15:00:34.695502 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1123/2255] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 15:00:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 15:00:34.703582 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021d10ac-61eb-4cb1-b24e-7f371e784739 None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:00:34.703957 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021d10ac-61eb-4cb1-b24e-7f371e784739 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:00:34.704299 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021d10ac-61eb-4cb1-b24e-7f371e784739 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:00:34.726580 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-021d10ac-61eb-4cb1-b24e-7f371e784739 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 15:00:34.727800 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-021d10ac-61eb-4cb1-b24e-7f371e784739 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:00:34.735793 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1133/2256] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 15:00:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 15:00:34.740107 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305c50b8-03d5-4aab-a3d6-42d10eda5244 None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:00:34.749929 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305c50b8-03d5-4aab-a3d6-42d10eda5244 None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:00:34.754881 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe69e93c-8613-4750-91aa-001a92147ae7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:00:34.755480 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe69e93c-8613-4750-91aa-001a92147ae7 admin admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:00:34.755760 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe69e93c-8613-4750-91aa-001a92147ae7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:00:34.756033 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe69e93c-8613-4750-91aa-001a92147ae7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:00:34.756681 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe69e93c-8613-4750-91aa-001a92147ae7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81341) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 15:00:34.756983 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe69e93c-8613-4750-91aa-001a92147ae7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:00:34.758625 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe69e93c-8613-4750-91aa-001a92147ae7 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:00:34.773361 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1124/2257] 10.222.0.33 () {58 vars in 1153 bytes} [Thu Sep 1 15:00:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 15:00:34.777553 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021d10ac-61eb-4cb1-b24e-7f371e784739 None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:00:34.786949 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021d10ac-61eb-4cb1-b24e-7f371e784739 None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:00:34.792369 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d1cee2-68e1-469e-8d4b-d5d90448cf5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:00:34.792960 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d1cee2-68e1-469e-8d4b-d5d90448cf5a admin admin] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:00:34.793238 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d1cee2-68e1-469e-8d4b-d5d90448cf5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:00:34.793509 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d1cee2-68e1-469e-8d4b-d5d90448cf5a admin admin] PATH_INFO: `/v3/projects` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:00:34.794149 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d1cee2-68e1-469e-8d4b-d5d90448cf5a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:00:34.796133 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d1cee2-68e1-469e-8d4b-d5d90448cf5a admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:00:34.824324 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1134/2258] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Sep 1 15:00:34 2022] POST /identity/v3/projects => generated 373 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 15:00:34.828451 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe69e93c-8613-4750-91aa-001a92147ae7 admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:00:34.839122 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe69e93c-8613-4750-91aa-001a92147ae7 admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:00:34.893266 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a80bf80-ffe0-4aa8-bbe9-99b88f174f9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:00:34.893864 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a80bf80-ffe0-4aa8-bbe9-99b88f174f9d admin admin] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:00:34.894149 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a80bf80-ffe0-4aa8-bbe9-99b88f174f9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:00:34.894561 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a80bf80-ffe0-4aa8-bbe9-99b88f174f9d admin admin] PATH_INFO: `/v3/users` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:00:34.895198 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a80bf80-ffe0-4aa8-bbe9-99b88f174f9d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:00:34.897774 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a80bf80-ffe0-4aa8-bbe9-99b88f174f9d admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:00:34.921078 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1125/2259] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Sep 1 15:00:34 2022] POST /identity/v3/users => generated 287 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 15:00:34.922548 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d1cee2-68e1-469e-8d4b-d5d90448cf5a admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:00:34.933034 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d1cee2-68e1-469e-8d4b-d5d90448cf5a admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:00:34.937749 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fbb676-0ecc-4d80-bcaf-e8ea480bbf46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:00:34.938459 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fbb676-0ecc-4d80-bcaf-e8ea480bbf46 admin admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:00:34.938738 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fbb676-0ecc-4d80-bcaf-e8ea480bbf46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:00:34.939005 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fbb676-0ecc-4d80-bcaf-e8ea480bbf46 admin admin] PATH_INFO: `/v3/roles` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:00:34.939642 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2fbb676-0ecc-4d80-bcaf-e8ea480bbf46 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:00:34.941243 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2fbb676-0ecc-4d80-bcaf-e8ea480bbf46 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:00:34.951528 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1135/2260] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 1 15:00:34 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 15:00:34.955365 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a80bf80-ffe0-4aa8-bbe9-99b88f174f9d admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:00:34.965832 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a80bf80-ffe0-4aa8-bbe9-99b88f174f9d admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:00:34.970508 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46ad213-7ed3-453a-a465-0535ede84e0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:00:34.971094 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46ad213-7ed3-453a-a465-0535ede84e0f admin admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:00:34.971373 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46ad213-7ed3-453a-a465-0535ede84e0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:00:34.971645 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46ad213-7ed3-453a-a465-0535ede84e0f admin admin] PATH_INFO: `/v3/roles` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:00:34.972287 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46ad213-7ed3-453a-a465-0535ede84e0f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:00:34.973919 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46ad213-7ed3-453a-a465-0535ede84e0f admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:00:34.984274 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1126/2261] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 1 15:00:34 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 15:00:34.988305 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fbb676-0ecc-4d80-bcaf-e8ea480bbf46 admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:00:34.998520 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fbb676-0ecc-4d80-bcaf-e8ea480bbf46 admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:00:35.003135 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102a1a57-b172-4a3f-896c-d0bf2ad1175e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:00:35.003784 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102a1a57-b172-4a3f-896c-d0bf2ad1175e admin admin] REQUEST_METHOD: `PUT` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:00:35.004061 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102a1a57-b172-4a3f-896c-d0bf2ad1175e admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:00:35.004334 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102a1a57-b172-4a3f-896c-d0bf2ad1175e admin admin] PATH_INFO: `/v3/projects/ba5a5ea6aa4b4736ae49915583626e18/users/534901c491b64bbe8bdc59776e643a78/roles/5d1feef0e13d4874928bd86258c4e3b2` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:00:35.019121 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102a1a57-b172-4a3f-896c-d0bf2ad1175e admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba5a5ea6aa4b4736ae49915583626e18, user_id=534901c491b64bbe8bdc59776e643a78, role_id=5d1feef0e13d4874928bd86258c4e3b2)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:00:35.022479 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102a1a57-b172-4a3f-896c-d0bf2ad1175e admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:00:35.036361 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1136/2262] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 1 15:00:34 2022] PUT /identity/v3/projects/ba5a5ea6aa4b4736ae49915583626e18/users/534901c491b64bbe8bdc59776e643a78/roles/5d1feef0e13d4874928bd86258c4e3b2 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 15:00:35.041411 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c32ee6-f21c-4ff7-a0e2-fbb1fc8196e8 None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:00:35.041870 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c32ee6-f21c-4ff7-a0e2-fbb1fc8196e8 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:00:35.042048 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c32ee6-f21c-4ff7-a0e2-fbb1fc8196e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:00:35.060796 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-13c32ee6-f21c-4ff7-a0e2-fbb1fc8196e8 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 15:00:35.062000 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-13c32ee6-f21c-4ff7-a0e2-fbb1fc8196e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:00:35.150608 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1127/2263] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 15:00:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 15:00:35.162374 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102a1a57-b172-4a3f-896c-d0bf2ad1175e admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:00:35.164093 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-102a1a57-b172-4a3f-896c-d0bf2ad1175e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:00:35.226655 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102a1a57-b172-4a3f-896c-d0bf2ad1175e admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:00:35.230365 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1136c73-0ad8-4075-9082-47315f678e15 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2790149639aa418bbc4e4982e93eda44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:00:35.230969 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1136c73-0ad8-4075-9082-47315f678e15 service neutron] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:00:35.231244 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1136c73-0ad8-4075-9082-47315f678e15 service neutron] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:00:35.231515 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1136c73-0ad8-4075-9082-47315f678e15 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:00:35.241404 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1136c73-0ad8-4075-9082-47315f678e15 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:00:35.242884 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1136c73-0ad8-4075-9082-47315f678e15 service neutron] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:00:35.252518 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1137/2264] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 15:00:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 15:00:36.090639 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768e4a0f-0761-417a-9afe-8c683cddbacf None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:00:36.091785 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768e4a0f-0761-417a-9afe-8c683cddbacf None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:00:36.092248 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768e4a0f-0761-417a-9afe-8c683cddbacf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:00:36.151039 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-768e4a0f-0761-417a-9afe-8c683cddbacf None None] MFA Rules not processed for user `534901c491b64bbe8bdc59776e643a78`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 15:00:36.152882 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-768e4a0f-0761-417a-9afe-8c683cddbacf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:00:36.180297 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 15:00:36.295365 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1128/2265] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 15:00:36 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 15:00:36.307459 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1136c73-0ad8-4075-9082-47315f678e15 service neutron] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:00:36.308874 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-f1136c73-0ad8-4075-9082-47315f678e15 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:00:36.315789 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 15:00:36.382078 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1136c73-0ad8-4075-9082-47315f678e15 service neutron] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:00:36.385948 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e62e964-33ae-4d81-a127-7fff6f42da94 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3b81739e34e4e648f88b6d73fcb5e46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:00:36.386830 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e62e964-33ae-4d81-a127-7fff6f42da94 service cinder] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:00:36.387069 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e62e964-33ae-4d81-a127-7fff6f42da94 service cinder] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:00:36.387340 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e62e964-33ae-4d81-a127-7fff6f42da94 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:00:36.399521 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e62e964-33ae-4d81-a127-7fff6f42da94 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:00:36.401001 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e62e964-33ae-4d81-a127-7fff6f42da94 service cinder] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:00:36.410185 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1138/2266] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 15:00:36 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 15:00:37.825494 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a67acc-822a-4be2-8dae-b59d26f03c37 None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:00:37.825984 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a67acc-822a-4be2-8dae-b59d26f03c37 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:00:37.826075 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a67acc-822a-4be2-8dae-b59d26f03c37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:00:37.844896 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-78a67acc-822a-4be2-8dae-b59d26f03c37 None None] MFA Rules not processed for user `534901c491b64bbe8bdc59776e643a78`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 15:00:37.846099 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-78a67acc-822a-4be2-8dae-b59d26f03c37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:00:37.853381 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1129/2267] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 15:00:37 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 15:00:46.406020 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=25 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 15:00:47.839475 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=22 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 15:00:51.847132 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e62e964-33ae-4d81-a127-7fff6f42da94 service cinder] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:00:51.848633 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-5e62e964-33ae-4d81-a127-7fff6f42da94 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:00:51.869954 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e62e964-33ae-4d81-a127-7fff6f42da94 service cinder] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:00:51.874608 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fc984f-3ba4-44a4-ab53-2d573af8209e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:00:51.875232 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fc984f-3ba4-44a4-ab53-2d573af8209e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:00:51.875505 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fc984f-3ba4-44a4-ab53-2d573af8209e admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:00:51.875898 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fc984f-3ba4-44a4-ab53-2d573af8209e admin admin] PATH_INFO: `/v3/users/7787af243c63452dadb2b9a957c43fda` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:00:51.877036 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68fc984f-3ba4-44a4-ab53-2d573af8209e admin admin] RBAC: Authorizing `identity:delete_user(user_id=7787af243c63452dadb2b9a957c43fda)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:00:51.879016 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68fc984f-3ba4-44a4-ab53-2d573af8209e admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:00:51.935612 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-68fc984f-3ba4-44a4-ab53-2d573af8209e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7787af243c63452dadb2b9a957c43fda'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:00:51.943830 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-68fc984f-3ba4-44a4-ab53-2d573af8209e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7787af243c63452dadb2b9a957c43fda'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:00:51.944694 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-68fc984f-3ba4-44a4-ab53-2d573af8209e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7787af243c63452dadb2b9a957c43fda'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:00:51.957523 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-68fc984f-3ba4-44a4-ab53-2d573af8209e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7787af243c63452dadb2b9a957c43fda'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:00:51.958285 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-68fc984f-3ba4-44a4-ab53-2d573af8209e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7787af243c63452dadb2b9a957c43fda'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:00:51.972120 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1139/2268] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Sep 1 15:00:51 2022] DELETE /identity/v3/users/7787af243c63452dadb2b9a957c43fda => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 15:00:51.977209 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5567538-134b-4755-9eaa-af16fe9bb908 None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:00:51.977734 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5567538-134b-4755-9eaa-af16fe9bb908 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:00:51.977734 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5567538-134b-4755-9eaa-af16fe9bb908 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:00:51.997455 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-d5567538-134b-4755-9eaa-af16fe9bb908 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 15:00:51.998716 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-d5567538-134b-4755-9eaa-af16fe9bb908 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:00:52.060433 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1130/2269] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 15:00:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 15:00:52.071523 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fc984f-3ba4-44a4-ab53-2d573af8209e admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:00:52.073042 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-68fc984f-3ba4-44a4-ab53-2d573af8209e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:00:52.136331 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fc984f-3ba4-44a4-ab53-2d573af8209e admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:00:52.141052 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f462b1-0d83-490a-a7e1-1aae94433754 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2790149639aa418bbc4e4982e93eda44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:00:52.141719 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f462b1-0d83-490a-a7e1-1aae94433754 service neutron] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:00:52.142009 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f462b1-0d83-490a-a7e1-1aae94433754 service neutron] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:00:52.142305 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f462b1-0d83-490a-a7e1-1aae94433754 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:00:52.152417 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f462b1-0d83-490a-a7e1-1aae94433754 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:00:52.153978 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f462b1-0d83-490a-a7e1-1aae94433754 service neutron] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:00:52.163636 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1140/2270] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 15:00:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 15:00:52.877248 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5567538-134b-4755-9eaa-af16fe9bb908 None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:00:52.878771 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-d5567538-134b-4755-9eaa-af16fe9bb908 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:00:52.900697 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5567538-134b-4755-9eaa-af16fe9bb908 None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:00:52.907390 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a828c4f5-8497-45bc-af91-383fc87ad3ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:00:52.908189 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a828c4f5-8497-45bc-af91-383fc87ad3ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:00:52.908393 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a828c4f5-8497-45bc-af91-383fc87ad3ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:00:52.908649 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a828c4f5-8497-45bc-af91-383fc87ad3ac admin admin] PATH_INFO: `/v3/projects/fba2b42d3ca24ba7901bc9e3cc2ed8b5` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:00:52.909910 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a828c4f5-8497-45bc-af91-383fc87ad3ac admin admin] RBAC: Authorizing `identity:delete_project(project_id=fba2b42d3ca24ba7901bc9e3cc2ed8b5)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:00:52.912993 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a828c4f5-8497-45bc-af91-383fc87ad3ac admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:00:52.973698 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-a828c4f5-8497-45bc-af91-383fc87ad3ac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fba2b42d3ca24ba7901bc9e3cc2ed8b5', 'request_id': 'req-a828c4f5-8497-45bc-af91-383fc87ad3ac'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:00:52.980954 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-a828c4f5-8497-45bc-af91-383fc87ad3ac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fba2b42d3ca24ba7901bc9e3cc2ed8b5', 'request_id': 'req-a828c4f5-8497-45bc-af91-383fc87ad3ac'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:00:52.991892 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-a828c4f5-8497-45bc-af91-383fc87ad3ac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fba2b42d3ca24ba7901bc9e3cc2ed8b5', 'request_id': 'req-a828c4f5-8497-45bc-af91-383fc87ad3ac'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:00:53.007916 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-a828c4f5-8497-45bc-af91-383fc87ad3ac admin admin] The token cache is being invalidate because project fba2b42d3ca24ba7901bc9e3cc2ed8b5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81341) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 15:00:53.008211 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-a828c4f5-8497-45bc-af91-383fc87ad3ac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:00:53.009042 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-a828c4f5-8497-45bc-af91-383fc87ad3ac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:00:53.010945 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1131/2271] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Sep 1 15:00:52 2022] DELETE /identity/v3/projects/fba2b42d3ca24ba7901bc9e3cc2ed8b5 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 15:00:53.015586 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3456195f-6a2e-49d9-b0bf-95bea77ebdbb None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:00:53.016074 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3456195f-6a2e-49d9-b0bf-95bea77ebdbb None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:00:53.016169 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3456195f-6a2e-49d9-b0bf-95bea77ebdbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:00:53.035935 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-3456195f-6a2e-49d9-b0bf-95bea77ebdbb None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 15:00:53.037153 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-3456195f-6a2e-49d9-b0bf-95bea77ebdbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:00:53.128722 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1141/2272] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 15:00:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 15:00:53.140651 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db38ffb-123e-46aa-80a1-669c0158a56f None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:00:53.140651 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db38ffb-123e-46aa-80a1-669c0158a56f None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:00:53.140651 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db38ffb-123e-46aa-80a1-669c0158a56f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:00:53.156944 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-0db38ffb-123e-46aa-80a1-669c0158a56f None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 15:00:53.158145 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-0db38ffb-123e-46aa-80a1-669c0158a56f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:00:53.166494 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1132/2273] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 15:00:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 15:00:53.170668 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3456195f-6a2e-49d9-b0bf-95bea77ebdbb None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:00:53.181034 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3456195f-6a2e-49d9-b0bf-95bea77ebdbb None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:00:53.185711 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d685f8-5bf5-42a0-808a-82eefdc87b3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:00:53.186421 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d685f8-5bf5-42a0-808a-82eefdc87b3a admin admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:00:53.186769 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d685f8-5bf5-42a0-808a-82eefdc87b3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:00:53.187131 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d685f8-5bf5-42a0-808a-82eefdc87b3a admin admin] PATH_INFO: `/v3/domains` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:00:53.187850 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d685f8-5bf5-42a0-808a-82eefdc87b3a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 15:00:53.188221 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d685f8-5bf5-42a0-808a-82eefdc87b3a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:00:53.189901 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d685f8-5bf5-42a0-808a-82eefdc87b3a admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:00:53.202048 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1142/2274] 10.222.0.33 () {58 vars in 1153 bytes} [Thu Sep 1 15:00:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 15:00:53.206567 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db38ffb-123e-46aa-80a1-669c0158a56f None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:00:53.216337 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db38ffb-123e-46aa-80a1-669c0158a56f None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:00:53.221312 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b9dbc3-776f-446d-a6eb-d1fde14521c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:00:53.221922 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b9dbc3-776f-446d-a6eb-d1fde14521c4 admin admin] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:00:53.222201 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b9dbc3-776f-446d-a6eb-d1fde14521c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:00:53.222503 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b9dbc3-776f-446d-a6eb-d1fde14521c4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:00:53.223153 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b9dbc3-776f-446d-a6eb-d1fde14521c4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:00:53.225789 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b9dbc3-776f-446d-a6eb-d1fde14521c4 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:00:53.254748 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1133/2275] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Sep 1 15:00:53 2022] POST /identity/v3/projects => generated 381 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 15:00:53.258939 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d685f8-5bf5-42a0-808a-82eefdc87b3a admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:00:53.264247 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 15:00:53.272462 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d685f8-5bf5-42a0-808a-82eefdc87b3a admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:00:53.332630 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6cb45c-f35e-487b-ac75-14d4cbedd8ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:00:53.333280 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6cb45c-f35e-487b-ac75-14d4cbedd8ac admin admin] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:00:53.333564 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6cb45c-f35e-487b-ac75-14d4cbedd8ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:00:53.333829 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6cb45c-f35e-487b-ac75-14d4cbedd8ac admin admin] PATH_INFO: `/v3/users` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:00:53.335137 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6cb45c-f35e-487b-ac75-14d4cbedd8ac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:00:53.337055 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6cb45c-f35e-487b-ac75-14d4cbedd8ac admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:00:53.347721 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1b0d6e-a502-4386-9127-f80a79c7b500 None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:00:53.348061 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1b0d6e-a502-4386-9127-f80a79c7b500 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:00:53.348519 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1b0d6e-a502-4386-9127-f80a79c7b500 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:00:53.359831 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1143/2276] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Sep 1 15:00:53 2022] POST /identity/v3/users => generated 291 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 15:00:53.364114 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6cb45c-f35e-487b-ac75-14d4cbedd8ac admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:00:53.368217 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-4f1b0d6e-a502-4386-9127-f80a79c7b500 None None] MFA Rules not processed for user `df75151c37f74b6da7428dc290a274ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 15:00:53.369384 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-4f1b0d6e-a502-4386-9127-f80a79c7b500 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:00:53.373503 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6cb45c-f35e-487b-ac75-14d4cbedd8ac admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:00:53.378058 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1caae2-96cb-4d07-9fc1-5aaa1ac3b791 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:00:53.378670 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1caae2-96cb-4d07-9fc1-5aaa1ac3b791 admin admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:00:53.378949 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1caae2-96cb-4d07-9fc1-5aaa1ac3b791 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:00:53.379216 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1caae2-96cb-4d07-9fc1-5aaa1ac3b791 admin admin] PATH_INFO: `/v3/roles` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:00:53.379849 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1caae2-96cb-4d07-9fc1-5aaa1ac3b791 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:00:53.381461 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1caae2-96cb-4d07-9fc1-5aaa1ac3b791 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:00:53.392159 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1144/2277] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 1 15:00:53 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 15:00:53.395971 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1caae2-96cb-4d07-9fc1-5aaa1ac3b791 admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:00:53.406554 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1caae2-96cb-4d07-9fc1-5aaa1ac3b791 admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:00:53.411307 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646773e8-38c5-4c36-b4b0-3e45fbfa2ead admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:00:53.411890 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646773e8-38c5-4c36-b4b0-3e45fbfa2ead admin admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:00:53.412171 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646773e8-38c5-4c36-b4b0-3e45fbfa2ead admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:00:53.413302 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646773e8-38c5-4c36-b4b0-3e45fbfa2ead admin admin] PATH_INFO: `/v3/roles` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:00:53.413682 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646773e8-38c5-4c36-b4b0-3e45fbfa2ead admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:00:53.415337 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646773e8-38c5-4c36-b4b0-3e45fbfa2ead admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:00:53.421034 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 15:00:53.425301 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1145/2278] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 1 15:00:53 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 15:00:53.430021 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646773e8-38c5-4c36-b4b0-3e45fbfa2ead admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:00:53.438393 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1134/2279] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 15:00:53 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 15:00:53.440110 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646773e8-38c5-4c36-b4b0-3e45fbfa2ead admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:00:53.444931 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0dfe537-644d-403b-8eae-2355a9217249 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:00:53.445573 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dfe537-644d-403b-8eae-2355a9217249 admin admin] REQUEST_METHOD: `PUT` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:00:53.445924 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dfe537-644d-403b-8eae-2355a9217249 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:00:53.446194 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dfe537-644d-403b-8eae-2355a9217249 admin admin] PATH_INFO: `/v3/projects/53f5d57b243e421cbf8d929cafe9c860/users/74b4bf3f69204fc896c996bcaa9e087e/roles/5d1feef0e13d4874928bd86258c4e3b2` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:00:53.462494 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0dfe537-644d-403b-8eae-2355a9217249 admin admin] RBAC: Authorizing `identity:create_grant(project_id=53f5d57b243e421cbf8d929cafe9c860, user_id=74b4bf3f69204fc896c996bcaa9e087e, role_id=5d1feef0e13d4874928bd86258c4e3b2)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:00:53.465139 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0dfe537-644d-403b-8eae-2355a9217249 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:00:53.479412 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1146/2280] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 1 15:00:53 2022] PUT /identity/v3/projects/53f5d57b243e421cbf8d929cafe9c860/users/74b4bf3f69204fc896c996bcaa9e087e/roles/5d1feef0e13d4874928bd86258c4e3b2 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 15:00:53.487437 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b54c3c0-c87e-42bb-8558-58d18bc2ab6a None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:00:53.488281 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b54c3c0-c87e-42bb-8558-58d18bc2ab6a None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:00:53.488567 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b54c3c0-c87e-42bb-8558-58d18bc2ab6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:00:53.510338 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-8b54c3c0-c87e-42bb-8558-58d18bc2ab6a None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 15:00:53.511608 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-8b54c3c0-c87e-42bb-8558-58d18bc2ab6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:00:53.573132 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1135/2281] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 15:00:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 15:00:53.582770 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0dfe537-644d-403b-8eae-2355a9217249 admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:00:53.584982 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-e0dfe537-644d-403b-8eae-2355a9217249 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:00:53.640933 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0dfe537-644d-403b-8eae-2355a9217249 admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:00:53.644463 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f56ce8-98bb-4c29-96dd-1f53a134029b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2790149639aa418bbc4e4982e93eda44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:00:53.645067 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f56ce8-98bb-4c29-96dd-1f53a134029b service neutron] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:00:53.645340 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f56ce8-98bb-4c29-96dd-1f53a134029b service neutron] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:00:53.645606 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f56ce8-98bb-4c29-96dd-1f53a134029b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:00:53.656928 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f56ce8-98bb-4c29-96dd-1f53a134029b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:00:53.658409 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f56ce8-98bb-4c29-96dd-1f53a134029b service neutron] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:00:53.668266 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1147/2282] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 15:00:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 15:00:54.615618 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eaddd5f-3263-4d70-ae65-abe94267e1ed None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:00:54.616612 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eaddd5f-3263-4d70-ae65-abe94267e1ed None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:00:54.617001 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eaddd5f-3263-4d70-ae65-abe94267e1ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:00:54.644726 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-9eaddd5f-3263-4d70-ae65-abe94267e1ed None None] MFA Rules not processed for user `74b4bf3f69204fc896c996bcaa9e087e`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 15:00:54.646055 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-9eaddd5f-3263-4d70-ae65-abe94267e1ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:00:54.722988 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1136/2283] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 15:00:54 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 15:00:54.734414 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f56ce8-98bb-4c29-96dd-1f53a134029b service neutron] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:00:54.735828 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-43f56ce8-98bb-4c29-96dd-1f53a134029b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:00:54.792034 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 15:00:54.797034 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f56ce8-98bb-4c29-96dd-1f53a134029b service neutron] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:00:54.802384 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5365e077-2d96-41b2-904e-c4cfc03fdc70 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3b81739e34e4e648f88b6d73fcb5e46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:00:54.803195 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5365e077-2d96-41b2-904e-c4cfc03fdc70 service cinder] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:00:54.803475 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5365e077-2d96-41b2-904e-c4cfc03fdc70 service cinder] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:00:54.803743 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5365e077-2d96-41b2-904e-c4cfc03fdc70 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:00:54.812382 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5365e077-2d96-41b2-904e-c4cfc03fdc70 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:00:54.813836 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5365e077-2d96-41b2-904e-c4cfc03fdc70 service cinder] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:00:54.822550 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1148/2284] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 15:00:54 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 15:00:56.171845 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b834af26-833a-48f8-8302-e472d1b51691 None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:00:56.172314 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b834af26-833a-48f8-8302-e472d1b51691 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:00:56.172419 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b834af26-833a-48f8-8302-e472d1b51691 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:00:56.191240 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-b834af26-833a-48f8-8302-e472d1b51691 None None] MFA Rules not processed for user `74b4bf3f69204fc896c996bcaa9e087e`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 15:00:56.192425 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-b834af26-833a-48f8-8302-e472d1b51691 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:00:56.199120 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1137/2285] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 15:00:56 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 15:00:59.150754 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5365e077-2d96-41b2-904e-c4cfc03fdc70 service cinder] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:00:59.151751 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-5365e077-2d96-41b2-904e-c4cfc03fdc70 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:00:59.180571 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5365e077-2d96-41b2-904e-c4cfc03fdc70 service cinder] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:00:59.186599 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bf27cf-fd6f-4034-9b3e-47217f0592a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:00:59.187242 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bf27cf-fd6f-4034-9b3e-47217f0592a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:00:59.187715 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bf27cf-fd6f-4034-9b3e-47217f0592a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:00:59.188734 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bf27cf-fd6f-4034-9b3e-47217f0592a5 admin admin] PATH_INFO: `/v3/users/b68a36f8bcb04650bd950a0e6b31ec3c` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:00:59.190422 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6bf27cf-fd6f-4034-9b3e-47217f0592a5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b68a36f8bcb04650bd950a0e6b31ec3c)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:00:59.193400 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6bf27cf-fd6f-4034-9b3e-47217f0592a5 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:00:59.257494 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-a6bf27cf-fd6f-4034-9b3e-47217f0592a5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b68a36f8bcb04650bd950a0e6b31ec3c'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:00:59.269989 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-a6bf27cf-fd6f-4034-9b3e-47217f0592a5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b68a36f8bcb04650bd950a0e6b31ec3c'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:00:59.271224 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-a6bf27cf-fd6f-4034-9b3e-47217f0592a5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b68a36f8bcb04650bd950a0e6b31ec3c'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:00:59.289775 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-a6bf27cf-fd6f-4034-9b3e-47217f0592a5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b68a36f8bcb04650bd950a0e6b31ec3c'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:00:59.290707 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-a6bf27cf-fd6f-4034-9b3e-47217f0592a5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b68a36f8bcb04650bd950a0e6b31ec3c'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:00:59.315346 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1149/2286] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Sep 1 15:00:59 2022] DELETE /identity/v3/users/b68a36f8bcb04650bd950a0e6b31ec3c => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 15:00:59.316827 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44311d9f-f461-48a4-8fa0-b561cc845eef None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:00:59.317671 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44311d9f-f461-48a4-8fa0-b561cc845eef None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:00:59.318194 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44311d9f-f461-48a4-8fa0-b561cc845eef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:00:59.353821 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-44311d9f-f461-48a4-8fa0-b561cc845eef None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 15:00:59.356462 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-44311d9f-f461-48a4-8fa0-b561cc845eef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:00:59.450084 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1138/2287] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 15:00:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 15:00:59.461160 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bf27cf-fd6f-4034-9b3e-47217f0592a5 admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:00:59.462644 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-a6bf27cf-fd6f-4034-9b3e-47217f0592a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:00:59.545363 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bf27cf-fd6f-4034-9b3e-47217f0592a5 admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:00:59.551957 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c0223b-8f48-4c1d-b121-6062543df276 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2790149639aa418bbc4e4982e93eda44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:00:59.551957 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c0223b-8f48-4c1d-b121-6062543df276 service neutron] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:00:59.551957 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c0223b-8f48-4c1d-b121-6062543df276 service neutron] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:00:59.551957 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c0223b-8f48-4c1d-b121-6062543df276 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:00:59.562930 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c0223b-8f48-4c1d-b121-6062543df276 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:00:59.564919 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c0223b-8f48-4c1d-b121-6062543df276 service neutron] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:00:59.583018 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1150/2288] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 15:00:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 15:00:59.829408 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aeb63b9-8ea7-405e-960e-44cbfa28c681 None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:00:59.829735 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aeb63b9-8ea7-405e-960e-44cbfa28c681 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:00:59.830058 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aeb63b9-8ea7-405e-960e-44cbfa28c681 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:00:59.853771 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-7aeb63b9-8ea7-405e-960e-44cbfa28c681 None None] MFA Rules not processed for user `24b6f275ffb04ffeae3c2097bfd040f5`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 15:00:59.855045 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-7aeb63b9-8ea7-405e-960e-44cbfa28c681 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:00:59.878884 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 15:00:59.938036 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1139/2289] 10.222.0.33 () {58 vars in 1009 bytes} [Thu Sep 1 15:00:59 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 15:00:59.989901 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c0223b-8f48-4c1d-b121-6062543df276 service neutron] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:00:59.992222 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-a6c0223b-8f48-4c1d-b121-6062543df276 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:01:00.031361 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aeb63b9-8ea7-405e-960e-44cbfa28c681 None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:01:00.034106 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-7aeb63b9-8ea7-405e-960e-44cbfa28c681 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:01:00.083053 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aeb63b9-8ea7-405e-960e-44cbfa28c681 None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:01:00.097372 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea15f3b-c166-4d5f-aceb-2f09c5bebb68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:01:00.098327 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea15f3b-c166-4d5f-aceb-2f09c5bebb68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:01:00.098765 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea15f3b-c166-4d5f-aceb-2f09c5bebb68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:01:00.099134 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea15f3b-c166-4d5f-aceb-2f09c5bebb68 admin admin] PATH_INFO: `/v3/projects/2d2592fec05a428aa804075ad3aa8cd3` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:01:00.102704 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea15f3b-c166-4d5f-aceb-2f09c5bebb68 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2d2592fec05a428aa804075ad3aa8cd3)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:01:00.105872 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea15f3b-c166-4d5f-aceb-2f09c5bebb68 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:01:00.170002 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c0223b-8f48-4c1d-b121-6062543df276 service neutron] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:01:00.181487 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c55f222-17c2-4654-a82f-f5b71c94ec44 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73a5bda5a6ea46e9863ccafd57dee339', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:01:00.182311 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c55f222-17c2-4654-a82f-f5b71c94ec44 service swift] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:01:00.182696 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c55f222-17c2-4654-a82f-f5b71c94ec44 service swift] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:01:00.183070 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c55f222-17c2-4654-a82f-f5b71c94ec44 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:01:00.190700 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 15:01:00.195632 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c55f222-17c2-4654-a82f-f5b71c94ec44 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:01:00.196807 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c55f222-17c2-4654-a82f-f5b71c94ec44 service swift] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:01:00.202938 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-bea15f3b-c166-4d5f-aceb-2f09c5bebb68 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2d2592fec05a428aa804075ad3aa8cd3', 'request_id': 'req-bea15f3b-c166-4d5f-aceb-2f09c5bebb68'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:01:00.211252 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-bea15f3b-c166-4d5f-aceb-2f09c5bebb68 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2d2592fec05a428aa804075ad3aa8cd3', 'request_id': 'req-bea15f3b-c166-4d5f-aceb-2f09c5bebb68'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:01:00.215228 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1151/2290] 10.222.0.33 () {60 vars in 1384 bytes} [Thu Sep 1 15:00:59 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 01 15:01:00.239802 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-bea15f3b-c166-4d5f-aceb-2f09c5bebb68 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2d2592fec05a428aa804075ad3aa8cd3', 'request_id': 'req-bea15f3b-c166-4d5f-aceb-2f09c5bebb68'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:01:00.254901 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-bea15f3b-c166-4d5f-aceb-2f09c5bebb68 admin admin] The token cache is being invalidate because project 2d2592fec05a428aa804075ad3aa8cd3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81341) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 15:01:00.255346 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-bea15f3b-c166-4d5f-aceb-2f09c5bebb68 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:01:00.257117 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-bea15f3b-c166-4d5f-aceb-2f09c5bebb68 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:01:00.259102 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1140/2291] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Sep 1 15:01:00 2022] DELETE /identity/v3/projects/2d2592fec05a428aa804075ad3aa8cd3 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 15:01:00.262939 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c55f222-17c2-4654-a82f-f5b71c94ec44 service swift] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:01:00.291575 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c55f222-17c2-4654-a82f-f5b71c94ec44 service swift] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:01:00.570587 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb75fd9-8ac6-4de5-bac2-83a8e42d1322 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:01:00.572563 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb75fd9-8ac6-4de5-bac2-83a8e42d1322 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:01:00.572855 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb75fd9-8ac6-4de5-bac2-83a8e42d1322 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:01:00.573194 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb75fd9-8ac6-4de5-bac2-83a8e42d1322 admin admin] PATH_INFO: `/v3/users/7cb20e8a8dd345aebd335d858bb77f9a` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:01:00.575161 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb75fd9-8ac6-4de5-bac2-83a8e42d1322 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7cb20e8a8dd345aebd335d858bb77f9a)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:01:00.577588 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb75fd9-8ac6-4de5-bac2-83a8e42d1322 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:01:00.708159 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-beb75fd9-8ac6-4de5-bac2-83a8e42d1322 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7cb20e8a8dd345aebd335d858bb77f9a'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:01:00.724615 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-beb75fd9-8ac6-4de5-bac2-83a8e42d1322 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7cb20e8a8dd345aebd335d858bb77f9a'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:01:00.726161 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-beb75fd9-8ac6-4de5-bac2-83a8e42d1322 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7cb20e8a8dd345aebd335d858bb77f9a'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:01:00.751522 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-beb75fd9-8ac6-4de5-bac2-83a8e42d1322 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7cb20e8a8dd345aebd335d858bb77f9a'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:01:00.753645 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-beb75fd9-8ac6-4de5-bac2-83a8e42d1322 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7cb20e8a8dd345aebd335d858bb77f9a'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:01:00.769752 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1152/2292] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Sep 1 15:01:00 2022] DELETE /identity/v3/users/7cb20e8a8dd345aebd335d858bb77f9a => generated 0 bytes in 508 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 15:01:01.244458 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea15f3b-c166-4d5f-aceb-2f09c5bebb68 admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:01:01.246609 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-bea15f3b-c166-4d5f-aceb-2f09c5bebb68 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:01:01.261051 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea15f3b-c166-4d5f-aceb-2f09c5bebb68 admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:01:01.326806 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ee51c3-b1b7-4f95-b28f-33276bc9463c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:01:01.327438 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ee51c3-b1b7-4f95-b28f-33276bc9463c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:01:01.327722 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ee51c3-b1b7-4f95-b28f-33276bc9463c admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:01:01.328000 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ee51c3-b1b7-4f95-b28f-33276bc9463c admin admin] PATH_INFO: `/v3/projects/ed703d1205ae4e3d9e5a29409b34329b` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:01:01.329268 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ee51c3-b1b7-4f95-b28f-33276bc9463c admin admin] RBAC: Authorizing `identity:delete_project(project_id=ed703d1205ae4e3d9e5a29409b34329b)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:01:01.331294 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ee51c3-b1b7-4f95-b28f-33276bc9463c admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:01:01.386844 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-83ee51c3-b1b7-4f95-b28f-33276bc9463c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ed703d1205ae4e3d9e5a29409b34329b', 'request_id': 'req-83ee51c3-b1b7-4f95-b28f-33276bc9463c'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:01:01.393462 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-83ee51c3-b1b7-4f95-b28f-33276bc9463c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ed703d1205ae4e3d9e5a29409b34329b', 'request_id': 'req-83ee51c3-b1b7-4f95-b28f-33276bc9463c'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:01:01.395391 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 15:01:01.407892 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-83ee51c3-b1b7-4f95-b28f-33276bc9463c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ed703d1205ae4e3d9e5a29409b34329b', 'request_id': 'req-83ee51c3-b1b7-4f95-b28f-33276bc9463c'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:01:01.428382 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-83ee51c3-b1b7-4f95-b28f-33276bc9463c admin admin] The token cache is being invalidate because project ed703d1205ae4e3d9e5a29409b34329b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81341) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 15:01:01.428671 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-83ee51c3-b1b7-4f95-b28f-33276bc9463c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:01:01.429328 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-83ee51c3-b1b7-4f95-b28f-33276bc9463c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:01:01.434378 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1141/2293] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Sep 1 15:01:01 2022] DELETE /identity/v3/projects/ed703d1205ae4e3d9e5a29409b34329b => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 15:01:01.436738 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c620b8-93e5-4993-824a-89436843e682 None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:01:01.436738 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c620b8-93e5-4993-824a-89436843e682 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:01:01.436738 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c620b8-93e5-4993-824a-89436843e682 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:01:01.459947 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-02c620b8-93e5-4993-824a-89436843e682 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 15:01:01.461620 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-02c620b8-93e5-4993-824a-89436843e682 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:01:01.550661 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da834292-6954-40d1-97be-20fb10c82430 None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:01:01.550855 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da834292-6954-40d1-97be-20fb10c82430 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:01:01.551378 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da834292-6954-40d1-97be-20fb10c82430 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:01:01.551648 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1153/2294] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 15:01:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 15:01:01.575618 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-da834292-6954-40d1-97be-20fb10c82430 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 15:01:01.577267 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-da834292-6954-40d1-97be-20fb10c82430 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:01:01.590419 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1142/2295] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 15:01:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 15:01:01.592513 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c620b8-93e5-4993-824a-89436843e682 None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:01:01.610019 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c620b8-93e5-4993-824a-89436843e682 None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:01:01.619862 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b2019e-cd41-4160-a760-afa78e2a970b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:01:01.620584 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b2019e-cd41-4160-a760-afa78e2a970b admin admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:01:01.620870 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b2019e-cd41-4160-a760-afa78e2a970b admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:01:01.621530 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b2019e-cd41-4160-a760-afa78e2a970b admin admin] PATH_INFO: `/v3/domains` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:01:01.622224 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b2019e-cd41-4160-a760-afa78e2a970b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 15:01:01.622885 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b2019e-cd41-4160-a760-afa78e2a970b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:01:01.625110 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b2019e-cd41-4160-a760-afa78e2a970b admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:01:01.642103 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1154/2296] 10.222.0.33 () {58 vars in 1153 bytes} [Thu Sep 1 15:01:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 15:01:01.643548 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da834292-6954-40d1-97be-20fb10c82430 None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:01:01.659501 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da834292-6954-40d1-97be-20fb10c82430 None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:01:01.671187 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ce45db-0c64-473b-9f64-a9d7f94cd600 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:01:01.672004 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ce45db-0c64-473b-9f64-a9d7f94cd600 admin admin] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:01:01.673234 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ce45db-0c64-473b-9f64-a9d7f94cd600 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:01:01.673792 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ce45db-0c64-473b-9f64-a9d7f94cd600 admin admin] PATH_INFO: `/v3/projects` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:01:01.674756 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ce45db-0c64-473b-9f64-a9d7f94cd600 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:01:01.677341 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ce45db-0c64-473b-9f64-a9d7f94cd600 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:01:01.741062 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1143/2297] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Sep 1 15:01:01 2022] POST /identity/v3/projects => generated 367 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 15:01:01.745640 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b2019e-cd41-4160-a760-afa78e2a970b admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:01:01.758402 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 15:01:01.766572 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b2019e-cd41-4160-a760-afa78e2a970b admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:01:01.888809 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdc3cdc-0832-4190-9a5f-adb989d73bf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:01:01.893361 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdc3cdc-0832-4190-9a5f-adb989d73bf9 admin admin] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:01:01.893820 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdc3cdc-0832-4190-9a5f-adb989d73bf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:01:01.894090 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdc3cdc-0832-4190-9a5f-adb989d73bf9 admin admin] PATH_INFO: `/v3/users` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:01:01.894793 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdc3cdc-0832-4190-9a5f-adb989d73bf9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:01:01.910459 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdc3cdc-0832-4190-9a5f-adb989d73bf9 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:01:01.936231 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1155/2298] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Sep 1 15:01:01 2022] POST /identity/v3/users => generated 284 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 15:01:01.943057 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ce45db-0c64-473b-9f64-a9d7f94cd600 admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:01:01.955014 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ce45db-0c64-473b-9f64-a9d7f94cd600 admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:01:01.963894 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b5ffe3-10a3-438b-84b8-bfe6d2c213cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:01:01.964514 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b5ffe3-10a3-438b-84b8-bfe6d2c213cd admin admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:01:01.964791 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b5ffe3-10a3-438b-84b8-bfe6d2c213cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:01:01.965066 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b5ffe3-10a3-438b-84b8-bfe6d2c213cd admin admin] PATH_INFO: `/v3/roles` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:01:01.965863 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b5ffe3-10a3-438b-84b8-bfe6d2c213cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:01:01.967673 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b5ffe3-10a3-438b-84b8-bfe6d2c213cd admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:01:01.984650 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1144/2299] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 1 15:01:01 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 15:01:01.988785 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdc3cdc-0832-4190-9a5f-adb989d73bf9 admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:01:02.001473 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdc3cdc-0832-4190-9a5f-adb989d73bf9 admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:01:02.008745 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7059188-b268-420d-9af9-90895c2216bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:01:02.009555 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7059188-b268-420d-9af9-90895c2216bb admin admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:01:02.009972 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7059188-b268-420d-9af9-90895c2216bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:01:02.010420 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7059188-b268-420d-9af9-90895c2216bb admin admin] PATH_INFO: `/v3/roles` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:01:02.011256 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7059188-b268-420d-9af9-90895c2216bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:01:02.013043 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7059188-b268-420d-9af9-90895c2216bb admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:01:02.040983 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1156/2300] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 1 15:01:01 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 15:01:02.045102 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b5ffe3-10a3-438b-84b8-bfe6d2c213cd admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:01:02.072228 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b5ffe3-10a3-438b-84b8-bfe6d2c213cd admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:01:02.169714 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9dbfa08-374b-4745-ab85-922c22f94bd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:01:02.171039 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dbfa08-374b-4745-ab85-922c22f94bd2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:01:02.171039 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dbfa08-374b-4745-ab85-922c22f94bd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:01:02.173061 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dbfa08-374b-4745-ab85-922c22f94bd2 admin admin] PATH_INFO: `/v3/projects/6070a3db75ab4f4a88e1d8ecda00d63e/users/1a634b1ef0434ac899691782b36db6b8/roles/5d1feef0e13d4874928bd86258c4e3b2` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:01:02.236556 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9dbfa08-374b-4745-ab85-922c22f94bd2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6070a3db75ab4f4a88e1d8ecda00d63e, user_id=1a634b1ef0434ac899691782b36db6b8, role_id=5d1feef0e13d4874928bd86258c4e3b2)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:01:02.239440 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9dbfa08-374b-4745-ab85-922c22f94bd2 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:01:02.265374 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1145/2301] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 1 15:01:02 2022] PUT /identity/v3/projects/6070a3db75ab4f4a88e1d8ecda00d63e/users/1a634b1ef0434ac899691782b36db6b8/roles/5d1feef0e13d4874928bd86258c4e3b2 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 15:01:02.276127 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fc98ba-7ae3-4469-b764-b38b5927877e None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:01:02.276414 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fc98ba-7ae3-4469-b764-b38b5927877e None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:01:02.276689 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fc98ba-7ae3-4469-b764-b38b5927877e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:01:02.318019 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-81fc98ba-7ae3-4469-b764-b38b5927877e None None] MFA Rules not processed for user `1a634b1ef0434ac899691782b36db6b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 15:01:02.319369 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-81fc98ba-7ae3-4469-b764-b38b5927877e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:01:02.422255 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1157/2302] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 15:01:02 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 15:01:02.427043 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9dbfa08-374b-4745-ab85-922c22f94bd2 admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:01:02.540095 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9dbfa08-374b-4745-ab85-922c22f94bd2 admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:01:02.547225 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7337d59-fea8-47c5-95e3-fde45328a774 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:01:02.548072 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7337d59-fea8-47c5-95e3-fde45328a774 admin admin] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:01:02.548362 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7337d59-fea8-47c5-95e3-fde45328a774 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:01:02.548635 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7337d59-fea8-47c5-95e3-fde45328a774 admin admin] PATH_INFO: `/v3/projects` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:01:02.549281 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7337d59-fea8-47c5-95e3-fde45328a774 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:01:02.551532 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7337d59-fea8-47c5-95e3-fde45328a774 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:01:02.590382 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1146/2303] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Sep 1 15:01:02 2022] POST /identity/v3/projects => generated 367 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 15:01:02.593056 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fc98ba-7ae3-4469-b764-b38b5927877e None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:01:02.605644 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fc98ba-7ae3-4469-b764-b38b5927877e None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:01:02.701542 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2483d6cb-bdb7-4b0f-b87a-fe88d2675bbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:01:02.702699 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2483d6cb-bdb7-4b0f-b87a-fe88d2675bbf admin admin] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:01:02.702699 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2483d6cb-bdb7-4b0f-b87a-fe88d2675bbf admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:01:02.702963 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2483d6cb-bdb7-4b0f-b87a-fe88d2675bbf admin admin] PATH_INFO: `/v3/users` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:01:02.703707 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2483d6cb-bdb7-4b0f-b87a-fe88d2675bbf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:01:02.705772 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2483d6cb-bdb7-4b0f-b87a-fe88d2675bbf admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:01:02.732678 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 15:01:02.743369 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1158/2304] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Sep 1 15:01:02 2022] POST /identity/v3/users => generated 290 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 15:01:02.747112 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7337d59-fea8-47c5-95e3-fde45328a774 admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:01:02.762384 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7337d59-fea8-47c5-95e3-fde45328a774 admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:01:02.768586 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875576d9-7ac9-4114-a9d8-9a1037982d68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:01:02.769317 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875576d9-7ac9-4114-a9d8-9a1037982d68 admin admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:01:02.769670 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875576d9-7ac9-4114-a9d8-9a1037982d68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:01:02.770027 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875576d9-7ac9-4114-a9d8-9a1037982d68 admin admin] PATH_INFO: `/v3/roles` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:01:02.770897 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875576d9-7ac9-4114-a9d8-9a1037982d68 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:01:02.772678 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875576d9-7ac9-4114-a9d8-9a1037982d68 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:01:02.784727 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1147/2305] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 1 15:01:02 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 15:01:02.789072 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2483d6cb-bdb7-4b0f-b87a-fe88d2675bbf admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:01:02.799712 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2483d6cb-bdb7-4b0f-b87a-fe88d2675bbf admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:01:02.805756 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ff3338-0834-4137-be93-b947a0bbe3eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:01:02.806547 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ff3338-0834-4137-be93-b947a0bbe3eb admin admin] REQUEST_METHOD: `PUT` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:01:02.806827 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ff3338-0834-4137-be93-b947a0bbe3eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:01:02.807173 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ff3338-0834-4137-be93-b947a0bbe3eb admin admin] PATH_INFO: `/v3/projects/4de35bb1e64240a495f172567a7d45b2/users/78313966560d46c08dd23083405c84fe/roles/2a2adb769e55476b82e5014897e0a9c0` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:01:02.828027 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ff3338-0834-4137-be93-b947a0bbe3eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=4de35bb1e64240a495f172567a7d45b2, user_id=78313966560d46c08dd23083405c84fe, role_id=2a2adb769e55476b82e5014897e0a9c0)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:01:02.833576 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ff3338-0834-4137-be93-b947a0bbe3eb admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:01:02.856795 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1159/2306] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 1 15:01:02 2022] PUT /identity/v3/projects/4de35bb1e64240a495f172567a7d45b2/users/78313966560d46c08dd23083405c84fe/roles/2a2adb769e55476b82e5014897e0a9c0 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 15:01:02.861459 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875576d9-7ac9-4114-a9d8-9a1037982d68 admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:01:02.877120 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875576d9-7ac9-4114-a9d8-9a1037982d68 admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:01:02.954746 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:INSERT=4,keystone:DELETE=3,keystone:SELECT=93 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 15:01:03.000735 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26547ab7-0123-4a0d-8b4d-76cffbec3c01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:01:03.000735 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26547ab7-0123-4a0d-8b4d-76cffbec3c01 admin admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:01:03.000735 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26547ab7-0123-4a0d-8b4d-76cffbec3c01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:01:03.000735 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26547ab7-0123-4a0d-8b4d-76cffbec3c01 admin admin] PATH_INFO: `/v3/roles` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:01:03.003141 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26547ab7-0123-4a0d-8b4d-76cffbec3c01 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:01:03.006807 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26547ab7-0123-4a0d-8b4d-76cffbec3c01 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:01:03.019638 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1148/2307] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 1 15:01:02 2022] GET /identity/v3/roles => generated 1420 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 15:01:03.025949 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ff3338-0834-4137-be93-b947a0bbe3eb admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:01:03.043327 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ff3338-0834-4137-be93-b947a0bbe3eb admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:01:03.065540 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79def72b-8832-4a5e-948f-4f52c0f4c188 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:01:03.070216 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79def72b-8832-4a5e-948f-4f52c0f4c188 admin admin] REQUEST_METHOD: `PUT` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:01:03.071873 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79def72b-8832-4a5e-948f-4f52c0f4c188 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:01:03.072308 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79def72b-8832-4a5e-948f-4f52c0f4c188 admin admin] PATH_INFO: `/v3/projects/4de35bb1e64240a495f172567a7d45b2/users/78313966560d46c08dd23083405c84fe/roles/2a2adb769e55476b82e5014897e0a9c0` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:01:03.075647 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79def72b-8832-4a5e-948f-4f52c0f4c188 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4de35bb1e64240a495f172567a7d45b2, user_id=78313966560d46c08dd23083405c84fe, role_id=2a2adb769e55476b82e5014897e0a9c0)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:01:03.100199 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79def72b-8832-4a5e-948f-4f52c0f4c188 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:01:03.115149 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1160/2308] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 1 15:01:03 2022] PUT /identity/v3/projects/4de35bb1e64240a495f172567a7d45b2/users/78313966560d46c08dd23083405c84fe/roles/2a2adb769e55476b82e5014897e0a9c0 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 15:01:03.127069 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a7ef98-d1ce-4c6c-a124-0f2e936c422c None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:01:03.127353 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a7ef98-d1ce-4c6c-a124-0f2e936c422c None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:01:03.127677 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a7ef98-d1ce-4c6c-a124-0f2e936c422c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:01:03.197507 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-86a7ef98-d1ce-4c6c-a124-0f2e936c422c None None] MFA Rules not processed for user `78313966560d46c08dd23083405c84fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 15:01:03.199087 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-86a7ef98-d1ce-4c6c-a124-0f2e936c422c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:01:03.328614 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1149/2309] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 15:01:03 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 15:01:03.339358 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79def72b-8832-4a5e-948f-4f52c0f4c188 admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:01:03.341047 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-79def72b-8832-4a5e-948f-4f52c0f4c188 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:01:03.451789 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79def72b-8832-4a5e-948f-4f52c0f4c188 admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:01:03.455698 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80a424c-d44f-4af2-b6b7-e829fa746f43 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3b81739e34e4e648f88b6d73fcb5e46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:01:03.456702 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80a424c-d44f-4af2-b6b7-e829fa746f43 service cinder] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:01:03.457078 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80a424c-d44f-4af2-b6b7-e829fa746f43 service cinder] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:01:03.457462 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80a424c-d44f-4af2-b6b7-e829fa746f43 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:01:03.476130 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80a424c-d44f-4af2-b6b7-e829fa746f43 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:01:03.478066 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80a424c-d44f-4af2-b6b7-e829fa746f43 service cinder] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:01:03.490344 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1161/2310] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 15:01:03 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 15:01:03.561369 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a7ef98-d1ce-4c6c-a124-0f2e936c422c None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:01:03.563424 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-86a7ef98-d1ce-4c6c-a124-0f2e936c422c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:01:03.577759 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a7ef98-d1ce-4c6c-a124-0f2e936c422c None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:01:03.581984 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c00690-1f07-4fd2-b82e-cda019a85d3e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3b81739e34e4e648f88b6d73fcb5e46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:01:03.585291 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c00690-1f07-4fd2-b82e-cda019a85d3e service cinder] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:01:03.586568 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c00690-1f07-4fd2-b82e-cda019a85d3e service cinder] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:01:03.587655 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c00690-1f07-4fd2-b82e-cda019a85d3e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:01:03.726866 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c00690-1f07-4fd2-b82e-cda019a85d3e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:01:03.728385 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c00690-1f07-4fd2-b82e-cda019a85d3e service cinder] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:01:03.766558 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1150/2311] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 15:01:03 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 15:01:07.841859 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80a424c-d44f-4af2-b6b7-e829fa746f43 service cinder] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:01:07.947767 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80a424c-d44f-4af2-b6b7-e829fa746f43 service cinder] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:01:07.954337 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acc09f2-eb33-401c-8c76-5de5a16164f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:01:07.955131 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acc09f2-eb33-401c-8c76-5de5a16164f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:01:07.955484 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acc09f2-eb33-401c-8c76-5de5a16164f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:01:07.955834 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acc09f2-eb33-401c-8c76-5de5a16164f7 admin admin] PATH_INFO: `/v3/users/1a634b1ef0434ac899691782b36db6b8` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:01:07.957578 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2acc09f2-eb33-401c-8c76-5de5a16164f7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1a634b1ef0434ac899691782b36db6b8)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:01:07.959956 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2acc09f2-eb33-401c-8c76-5de5a16164f7 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:01:08.046618 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-2acc09f2-eb33-401c-8c76-5de5a16164f7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1a634b1ef0434ac899691782b36db6b8'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:01:08.059926 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-2acc09f2-eb33-401c-8c76-5de5a16164f7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1a634b1ef0434ac899691782b36db6b8'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:01:08.061807 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-2acc09f2-eb33-401c-8c76-5de5a16164f7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1a634b1ef0434ac899691782b36db6b8'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:01:08.086565 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 15:01:08.091371 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-2acc09f2-eb33-401c-8c76-5de5a16164f7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1a634b1ef0434ac899691782b36db6b8'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:01:08.096215 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-2acc09f2-eb33-401c-8c76-5de5a16164f7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1a634b1ef0434ac899691782b36db6b8'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:01:08.126139 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1162/2312] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Sep 1 15:01:07 2022] DELETE /identity/v3/users/1a634b1ef0434ac899691782b36db6b8 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 15:01:08.127837 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e7eaa8-0f58-4137-8c56-b474b5eb70de None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:01:08.128361 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e7eaa8-0f58-4137-8c56-b474b5eb70de None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:01:08.128464 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e7eaa8-0f58-4137-8c56-b474b5eb70de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:01:08.152172 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-d6e7eaa8-0f58-4137-8c56-b474b5eb70de None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 15:01:08.154880 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-d6e7eaa8-0f58-4137-8c56-b474b5eb70de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:01:08.292653 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1151/2313] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 15:01:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 15:01:08.326197 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acc09f2-eb33-401c-8c76-5de5a16164f7 admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:01:08.327990 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-2acc09f2-eb33-401c-8c76-5de5a16164f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:01:08.500479 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acc09f2-eb33-401c-8c76-5de5a16164f7 admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:01:08.518623 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec00d49-2136-4fb3-8a0f-17deb70d7751 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2790149639aa418bbc4e4982e93eda44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:01:08.522223 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec00d49-2136-4fb3-8a0f-17deb70d7751 service neutron] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:01:08.522726 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec00d49-2136-4fb3-8a0f-17deb70d7751 service neutron] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:01:08.523413 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec00d49-2136-4fb3-8a0f-17deb70d7751 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:01:08.546957 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec00d49-2136-4fb3-8a0f-17deb70d7751 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:01:08.548801 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec00d49-2136-4fb3-8a0f-17deb70d7751 service neutron] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:01:08.584141 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1163/2314] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 15:01:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 253 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 15:01:09.170104 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e7eaa8-0f58-4137-8c56-b474b5eb70de None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:01:09.171871 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-d6e7eaa8-0f58-4137-8c56-b474b5eb70de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:01:09.195244 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e7eaa8-0f58-4137-8c56-b474b5eb70de None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:01:09.200373 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b84a87c-ae2a-4241-a8ae-ece28a1604ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:01:09.201796 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b84a87c-ae2a-4241-a8ae-ece28a1604ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:01:09.202377 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b84a87c-ae2a-4241-a8ae-ece28a1604ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:01:09.203013 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b84a87c-ae2a-4241-a8ae-ece28a1604ef admin admin] PATH_INFO: `/v3/projects/6070a3db75ab4f4a88e1d8ecda00d63e` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:01:09.205298 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b84a87c-ae2a-4241-a8ae-ece28a1604ef admin admin] RBAC: Authorizing `identity:delete_project(project_id=6070a3db75ab4f4a88e1d8ecda00d63e)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:01:09.208306 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b84a87c-ae2a-4241-a8ae-ece28a1604ef admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:01:09.218126 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 15:01:09.271295 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-0b84a87c-ae2a-4241-a8ae-ece28a1604ef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6070a3db75ab4f4a88e1d8ecda00d63e', 'request_id': 'req-0b84a87c-ae2a-4241-a8ae-ece28a1604ef'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:01:09.278703 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-0b84a87c-ae2a-4241-a8ae-ece28a1604ef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6070a3db75ab4f4a88e1d8ecda00d63e', 'request_id': 'req-0b84a87c-ae2a-4241-a8ae-ece28a1604ef'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:01:09.288222 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-0b84a87c-ae2a-4241-a8ae-ece28a1604ef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6070a3db75ab4f4a88e1d8ecda00d63e', 'request_id': 'req-0b84a87c-ae2a-4241-a8ae-ece28a1604ef'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:01:09.302082 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-0b84a87c-ae2a-4241-a8ae-ece28a1604ef admin admin] The token cache is being invalidate because project 6070a3db75ab4f4a88e1d8ecda00d63e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81341) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 15:01:09.302514 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-0b84a87c-ae2a-4241-a8ae-ece28a1604ef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:01:09.303800 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-0b84a87c-ae2a-4241-a8ae-ece28a1604ef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:01:09.308404 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1152/2315] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Sep 1 15:01:09 2022] DELETE /identity/v3/projects/6070a3db75ab4f4a88e1d8ecda00d63e => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 15:01:09.309537 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec00d49-2136-4fb3-8a0f-17deb70d7751 service neutron] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:01:09.321341 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec00d49-2136-4fb3-8a0f-17deb70d7751 service neutron] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:01:09.398344 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa03184-7fbd-4bde-a234-afce847f3112 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:01:09.398989 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa03184-7fbd-4bde-a234-afce847f3112 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:01:09.399270 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa03184-7fbd-4bde-a234-afce847f3112 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:01:09.399657 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa03184-7fbd-4bde-a234-afce847f3112 admin admin] PATH_INFO: `/v3/users/78313966560d46c08dd23083405c84fe` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:01:09.404517 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa03184-7fbd-4bde-a234-afce847f3112 admin admin] RBAC: Authorizing `identity:delete_user(user_id=78313966560d46c08dd23083405c84fe)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:01:09.406695 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa03184-7fbd-4bde-a234-afce847f3112 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:01:09.466131 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-dfa03184-7fbd-4bde-a234-afce847f3112 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '78313966560d46c08dd23083405c84fe'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:01:09.477134 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-dfa03184-7fbd-4bde-a234-afce847f3112 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '78313966560d46c08dd23083405c84fe'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:01:09.477762 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-dfa03184-7fbd-4bde-a234-afce847f3112 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '78313966560d46c08dd23083405c84fe'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:01:09.492852 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-dfa03184-7fbd-4bde-a234-afce847f3112 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '78313966560d46c08dd23083405c84fe'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:01:09.493722 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-dfa03184-7fbd-4bde-a234-afce847f3112 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '78313966560d46c08dd23083405c84fe'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:01:09.515166 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1164/2316] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Sep 1 15:01:09 2022] DELETE /identity/v3/users/78313966560d46c08dd23083405c84fe => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 15:01:10.287205 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b84a87c-ae2a-4241-a8ae-ece28a1604ef admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:01:10.289495 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-0b84a87c-ae2a-4241-a8ae-ece28a1604ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:01:10.308855 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b84a87c-ae2a-4241-a8ae-ece28a1604ef admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:01:10.379584 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849d5af0-f931-4aea-8f00-ee564b7cc534 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:01:10.379584 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849d5af0-f931-4aea-8f00-ee564b7cc534 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:01:10.381434 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849d5af0-f931-4aea-8f00-ee564b7cc534 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:01:10.381681 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849d5af0-f931-4aea-8f00-ee564b7cc534 admin admin] PATH_INFO: `/v3/projects/4de35bb1e64240a495f172567a7d45b2` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:01:10.382999 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849d5af0-f931-4aea-8f00-ee564b7cc534 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4de35bb1e64240a495f172567a7d45b2)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:01:10.384989 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849d5af0-f931-4aea-8f00-ee564b7cc534 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:01:10.469870 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-849d5af0-f931-4aea-8f00-ee564b7cc534 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4de35bb1e64240a495f172567a7d45b2', 'request_id': 'req-849d5af0-f931-4aea-8f00-ee564b7cc534'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:01:10.485568 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-849d5af0-f931-4aea-8f00-ee564b7cc534 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4de35bb1e64240a495f172567a7d45b2', 'request_id': 'req-849d5af0-f931-4aea-8f00-ee564b7cc534'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:01:10.508801 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-849d5af0-f931-4aea-8f00-ee564b7cc534 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4de35bb1e64240a495f172567a7d45b2', 'request_id': 'req-849d5af0-f931-4aea-8f00-ee564b7cc534'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:01:10.532637 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-849d5af0-f931-4aea-8f00-ee564b7cc534 admin admin] The token cache is being invalidate because project 4de35bb1e64240a495f172567a7d45b2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81341) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 15:01:10.532978 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-849d5af0-f931-4aea-8f00-ee564b7cc534 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:01:10.533643 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-849d5af0-f931-4aea-8f00-ee564b7cc534 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:01:10.538818 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1153/2317] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Sep 1 15:01:10 2022] DELETE /identity/v3/projects/4de35bb1e64240a495f172567a7d45b2 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 15:01:10.540556 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b733c30-e04e-47fb-9016-503f4c94dce2 None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:01:10.541044 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b733c30-e04e-47fb-9016-503f4c94dce2 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:01:10.541158 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b733c30-e04e-47fb-9016-503f4c94dce2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:01:10.557753 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 15:01:10.566486 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-0b733c30-e04e-47fb-9016-503f4c94dce2 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 15:01:10.569346 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-0b733c30-e04e-47fb-9016-503f4c94dce2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:01:10.658633 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1165/2318] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 15:01:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 15:01:10.672413 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe69f7a2-07a4-43fe-b811-0cbc1d4e1816 None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:01:10.672818 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe69f7a2-07a4-43fe-b811-0cbc1d4e1816 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:01:10.673587 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe69f7a2-07a4-43fe-b811-0cbc1d4e1816 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:01:10.718315 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-fe69f7a2-07a4-43fe-b811-0cbc1d4e1816 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 15:01:10.721202 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-fe69f7a2-07a4-43fe-b811-0cbc1d4e1816 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:01:10.744342 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1154/2319] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 15:01:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 15:01:10.749646 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b733c30-e04e-47fb-9016-503f4c94dce2 None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:01:10.763935 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b733c30-e04e-47fb-9016-503f4c94dce2 None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:01:10.770653 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0f8380-3940-4280-83ea-6a29f57e9432 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:01:10.771358 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0f8380-3940-4280-83ea-6a29f57e9432 admin admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:01:10.771804 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0f8380-3940-4280-83ea-6a29f57e9432 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:01:10.772074 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0f8380-3940-4280-83ea-6a29f57e9432 admin admin] PATH_INFO: `/v3/domains` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:01:10.773008 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0f8380-3940-4280-83ea-6a29f57e9432 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 15:01:10.773320 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0f8380-3940-4280-83ea-6a29f57e9432 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:01:10.775038 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0f8380-3940-4280-83ea-6a29f57e9432 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:01:10.796507 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1166/2320] 10.222.0.33 () {58 vars in 1153 bytes} [Thu Sep 1 15:01:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 15:01:10.797274 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe69f7a2-07a4-43fe-b811-0cbc1d4e1816 None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:01:10.812818 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe69f7a2-07a4-43fe-b811-0cbc1d4e1816 None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:01:10.824167 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ed3159-533e-422f-a699-1653b9da3a29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:01:10.825003 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ed3159-533e-422f-a699-1653b9da3a29 admin admin] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:01:10.826118 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ed3159-533e-422f-a699-1653b9da3a29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:01:10.826530 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ed3159-533e-422f-a699-1653b9da3a29 admin admin] PATH_INFO: `/v3/projects` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:01:10.827442 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ed3159-533e-422f-a699-1653b9da3a29 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:01:10.831040 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ed3159-533e-422f-a699-1653b9da3a29 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:01:10.882895 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 15:01:10.894517 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0f8380-3940-4280-83ea-6a29f57e9432 admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:01:10.899841 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1155/2321] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Sep 1 15:01:10 2022] POST /identity/v3/projects => generated 387 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 15:01:10.908208 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0f8380-3940-4280-83ea-6a29f57e9432 admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:01:11.016253 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d6fddd-039c-4b52-aca6-47f87ecbeb57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:01:11.017565 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d6fddd-039c-4b52-aca6-47f87ecbeb57 admin admin] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:01:11.018311 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d6fddd-039c-4b52-aca6-47f87ecbeb57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:01:11.018668 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d6fddd-039c-4b52-aca6-47f87ecbeb57 admin admin] PATH_INFO: `/v3/users` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:01:11.019373 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d6fddd-039c-4b52-aca6-47f87ecbeb57 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:01:11.022041 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d6fddd-039c-4b52-aca6-47f87ecbeb57 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:01:11.052150 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1167/2322] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Sep 1 15:01:10 2022] POST /identity/v3/users => generated 294 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 15:01:11.056258 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ed3159-533e-422f-a699-1653b9da3a29 admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:01:11.072605 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ed3159-533e-422f-a699-1653b9da3a29 admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:01:11.087369 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549ae35c-9e73-4009-8c00-aff8d9b1dbf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:01:11.087996 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549ae35c-9e73-4009-8c00-aff8d9b1dbf0 admin admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:01:11.088273 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549ae35c-9e73-4009-8c00-aff8d9b1dbf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:01:11.088550 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549ae35c-9e73-4009-8c00-aff8d9b1dbf0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:01:11.089202 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549ae35c-9e73-4009-8c00-aff8d9b1dbf0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:01:11.090938 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549ae35c-9e73-4009-8c00-aff8d9b1dbf0 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:01:11.111461 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1156/2323] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 1 15:01:11 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 15:01:11.114275 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d6fddd-039c-4b52-aca6-47f87ecbeb57 admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:01:11.128323 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d6fddd-039c-4b52-aca6-47f87ecbeb57 admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:01:11.134877 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a1cdff-8de7-4120-bcac-a05cacdd98cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:01:11.135766 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a1cdff-8de7-4120-bcac-a05cacdd98cf admin admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:01:11.136308 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a1cdff-8de7-4120-bcac-a05cacdd98cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:01:11.136780 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a1cdff-8de7-4120-bcac-a05cacdd98cf admin admin] PATH_INFO: `/v3/roles` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:01:11.137965 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a1cdff-8de7-4120-bcac-a05cacdd98cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:01:11.139893 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a1cdff-8de7-4120-bcac-a05cacdd98cf admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:01:11.164800 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1168/2324] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 1 15:01:11 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 15:01:11.169323 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549ae35c-9e73-4009-8c00-aff8d9b1dbf0 admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:01:11.202259 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549ae35c-9e73-4009-8c00-aff8d9b1dbf0 admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:01:11.213863 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c6dd41-c8ed-4007-b0ae-e27829aa896c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:01:11.215807 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c6dd41-c8ed-4007-b0ae-e27829aa896c admin admin] REQUEST_METHOD: `PUT` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:01:11.216014 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c6dd41-c8ed-4007-b0ae-e27829aa896c admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:01:11.216446 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c6dd41-c8ed-4007-b0ae-e27829aa896c admin admin] PATH_INFO: `/v3/projects/42f317e30f0945928fabbcad65551c87/users/d869e6cc78424763bad6a2bb76996f85/roles/5d1feef0e13d4874928bd86258c4e3b2` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:01:11.256316 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c6dd41-c8ed-4007-b0ae-e27829aa896c admin admin] RBAC: Authorizing `identity:create_grant(project_id=42f317e30f0945928fabbcad65551c87, user_id=d869e6cc78424763bad6a2bb76996f85, role_id=5d1feef0e13d4874928bd86258c4e3b2)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:01:11.259735 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c6dd41-c8ed-4007-b0ae-e27829aa896c admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:01:11.291757 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1157/2325] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 1 15:01:11 2022] PUT /identity/v3/projects/42f317e30f0945928fabbcad65551c87/users/d869e6cc78424763bad6a2bb76996f85/roles/5d1feef0e13d4874928bd86258c4e3b2 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 15:01:11.302151 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f443d0-979d-449a-a907-c3a809c0f547 None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:01:11.308049 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f443d0-979d-449a-a907-c3a809c0f547 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:01:11.309105 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f443d0-979d-449a-a907-c3a809c0f547 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:01:11.384777 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-24f443d0-979d-449a-a907-c3a809c0f547 None None] MFA Rules not processed for user `d869e6cc78424763bad6a2bb76996f85`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 15:01:11.384777 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-24f443d0-979d-449a-a907-c3a809c0f547 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:01:11.559790 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c6dd41-c8ed-4007-b0ae-e27829aa896c admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:01:11.568594 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1169/2326] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 15:01:11 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 15:01:11.689218 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c6dd41-c8ed-4007-b0ae-e27829aa896c admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:01:11.694932 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1eeb58-352d-4058-96ca-2c0fa104ae5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:01:11.695537 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1eeb58-352d-4058-96ca-2c0fa104ae5d admin admin] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:01:11.695818 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1eeb58-352d-4058-96ca-2c0fa104ae5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:01:11.696667 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1eeb58-352d-4058-96ca-2c0fa104ae5d admin admin] PATH_INFO: `/v3/projects` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:01:11.697368 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1eeb58-352d-4058-96ca-2c0fa104ae5d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:01:11.700689 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1eeb58-352d-4058-96ca-2c0fa104ae5d admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:01:11.744152 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1158/2327] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Sep 1 15:01:11 2022] POST /identity/v3/projects => generated 385 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 15:01:11.747912 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f443d0-979d-449a-a907-c3a809c0f547 None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:01:11.754176 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 15:01:11.762844 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f443d0-979d-449a-a907-c3a809c0f547 None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:01:11.765932 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1cb75a-ba72-4f65-9266-a2ba793df478 None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:01:11.765932 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1cb75a-ba72-4f65-9266-a2ba793df478 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:01:11.765932 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1cb75a-ba72-4f65-9266-a2ba793df478 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:01:11.796036 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-5b1cb75a-ba72-4f65-9266-a2ba793df478 None None] MFA Rules not processed for user `534901c491b64bbe8bdc59776e643a78`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 15:01:11.797264 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-5b1cb75a-ba72-4f65-9266-a2ba793df478 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:01:11.866901 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1159/2328] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 15:01:11 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 15:01:11.932187 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35236f5e-a87d-4e5b-869c-5c5c087ddb3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:01:11.933061 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35236f5e-a87d-4e5b-869c-5c5c087ddb3e admin admin] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:01:11.933628 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35236f5e-a87d-4e5b-869c-5c5c087ddb3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:01:11.934057 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35236f5e-a87d-4e5b-869c-5c5c087ddb3e admin admin] PATH_INFO: `/v3/users` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:01:11.935103 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35236f5e-a87d-4e5b-869c-5c5c087ddb3e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:01:11.937334 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35236f5e-a87d-4e5b-869c-5c5c087ddb3e admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:01:11.982925 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1170/2329] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Sep 1 15:01:11 2022] POST /identity/v3/users => generated 293 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 15:01:11.983874 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1cb75a-ba72-4f65-9266-a2ba793df478 None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:01:12.000945 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1cb75a-ba72-4f65-9266-a2ba793df478 None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:01:12.020311 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa16d69f-da77-4f0f-bdab-214415cfb128 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:01:12.020743 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa16d69f-da77-4f0f-bdab-214415cfb128 admin admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:01:12.021017 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa16d69f-da77-4f0f-bdab-214415cfb128 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:01:12.021317 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa16d69f-da77-4f0f-bdab-214415cfb128 admin admin] PATH_INFO: `/v3/roles` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:01:12.022122 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa16d69f-da77-4f0f-bdab-214415cfb128 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:01:12.024268 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa16d69f-da77-4f0f-bdab-214415cfb128 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:01:12.045049 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35236f5e-a87d-4e5b-869c-5c5c087ddb3e admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:01:12.046407 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1160/2330] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 1 15:01:11 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 15:01:12.056247 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35236f5e-a87d-4e5b-869c-5c5c087ddb3e admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:01:12.062174 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3429c00b-7df0-4345-b9bc-bf3c5281cb93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:01:12.062970 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3429c00b-7df0-4345-b9bc-bf3c5281cb93 admin admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:01:12.063454 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3429c00b-7df0-4345-b9bc-bf3c5281cb93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:01:12.063881 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3429c00b-7df0-4345-b9bc-bf3c5281cb93 admin admin] PATH_INFO: `/v3/roles` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:01:12.064682 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3429c00b-7df0-4345-b9bc-bf3c5281cb93 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:01:12.066585 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3429c00b-7df0-4345-b9bc-bf3c5281cb93 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:01:12.086670 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1171/2331] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 1 15:01:12 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 15:01:12.087389 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa16d69f-da77-4f0f-bdab-214415cfb128 admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:01:12.109224 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa16d69f-da77-4f0f-bdab-214415cfb128 admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:01:12.117396 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5094e2-d1c7-4071-a70d-4e53671a57cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:01:12.118621 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5094e2-d1c7-4071-a70d-4e53671a57cf admin admin] REQUEST_METHOD: `PUT` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:01:12.119162 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5094e2-d1c7-4071-a70d-4e53671a57cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:01:12.119647 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5094e2-d1c7-4071-a70d-4e53671a57cf admin admin] PATH_INFO: `/v3/projects/9595ebf6550f43eb80b0a716c466644e/users/0ee79dda1c8c4ec8ad0e38a5f8b296c1/roles/5d1feef0e13d4874928bd86258c4e3b2` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:01:12.145515 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5094e2-d1c7-4071-a70d-4e53671a57cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=9595ebf6550f43eb80b0a716c466644e, user_id=0ee79dda1c8c4ec8ad0e38a5f8b296c1, role_id=5d1feef0e13d4874928bd86258c4e3b2)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:01:12.149615 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5094e2-d1c7-4071-a70d-4e53671a57cf admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:01:12.186442 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1161/2332] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 1 15:01:12 2022] PUT /identity/v3/projects/9595ebf6550f43eb80b0a716c466644e/users/0ee79dda1c8c4ec8ad0e38a5f8b296c1/roles/5d1feef0e13d4874928bd86258c4e3b2 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 15:01:12.197819 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e4d98a-3ffc-4220-801d-166aa09b82e4 None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:01:12.198189 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e4d98a-3ffc-4220-801d-166aa09b82e4 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:01:12.198560 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e4d98a-3ffc-4220-801d-166aa09b82e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:01:12.251736 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-41e4d98a-3ffc-4220-801d-166aa09b82e4 None None] MFA Rules not processed for user `0ee79dda1c8c4ec8ad0e38a5f8b296c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 15:01:12.254759 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-41e4d98a-3ffc-4220-801d-166aa09b82e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:01:12.385884 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1172/2333] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 15:01:12 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 15:01:12.391944 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5094e2-d1c7-4071-a70d-4e53671a57cf admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:01:12.434107 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 15:01:12.526138 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5094e2-d1c7-4071-a70d-4e53671a57cf admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:01:12.548084 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8bfa40-a385-4830-83dc-0aa7350e1f26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:01:12.550451 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8bfa40-a385-4830-83dc-0aa7350e1f26 admin admin] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:01:12.551313 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8bfa40-a385-4830-83dc-0aa7350e1f26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:01:12.551612 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8bfa40-a385-4830-83dc-0aa7350e1f26 admin admin] PATH_INFO: `/v3/projects` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:01:12.553495 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8bfa40-a385-4830-83dc-0aa7350e1f26 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:01:12.556590 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8bfa40-a385-4830-83dc-0aa7350e1f26 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:01:12.616136 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1162/2334] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Sep 1 15:01:12 2022] POST /identity/v3/projects => generated 387 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 15:01:12.619192 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e4d98a-3ffc-4220-801d-166aa09b82e4 None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:01:12.635915 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e4d98a-3ffc-4220-801d-166aa09b82e4 None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:01:12.804392 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43da573-a9d0-460b-b313-564a291a9a22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:01:12.805032 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43da573-a9d0-460b-b313-564a291a9a22 admin admin] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:01:12.805499 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43da573-a9d0-460b-b313-564a291a9a22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:01:12.805779 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43da573-a9d0-460b-b313-564a291a9a22 admin admin] PATH_INFO: `/v3/users` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:01:12.806475 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43da573-a9d0-460b-b313-564a291a9a22 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:01:12.808492 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43da573-a9d0-460b-b313-564a291a9a22 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:01:12.859915 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 15:01:12.887611 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1173/2335] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Sep 1 15:01:12 2022] POST /identity/v3/users => generated 300 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 15:01:12.888461 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8bfa40-a385-4830-83dc-0aa7350e1f26 admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:01:12.903768 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8bfa40-a385-4830-83dc-0aa7350e1f26 admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:01:12.922953 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82acd02-20dd-4250-a495-a654d0f8e7d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:01:12.926449 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82acd02-20dd-4250-a495-a654d0f8e7d7 admin admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:01:12.926974 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82acd02-20dd-4250-a495-a654d0f8e7d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:01:12.927451 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82acd02-20dd-4250-a495-a654d0f8e7d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:01:12.928792 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82acd02-20dd-4250-a495-a654d0f8e7d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:01:12.931642 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82acd02-20dd-4250-a495-a654d0f8e7d7 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:01:12.949260 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1163/2336] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 1 15:01:12 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 15:01:12.950416 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43da573-a9d0-460b-b313-564a291a9a22 admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:01:12.966592 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43da573-a9d0-460b-b313-564a291a9a22 admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:01:12.972044 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930a2c48-30df-4014-a064-369cfcb36056 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:01:12.972710 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930a2c48-30df-4014-a064-369cfcb36056 admin admin] REQUEST_METHOD: `PUT` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:01:12.973203 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930a2c48-30df-4014-a064-369cfcb36056 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:01:12.973552 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930a2c48-30df-4014-a064-369cfcb36056 admin admin] PATH_INFO: `/v3/projects/9962d041e4c84eee8a16348db69b92b9/users/0d74be5e44544f0b903451665c2122fb/roles/2a2adb769e55476b82e5014897e0a9c0` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:01:12.993561 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930a2c48-30df-4014-a064-369cfcb36056 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9962d041e4c84eee8a16348db69b92b9, user_id=0d74be5e44544f0b903451665c2122fb, role_id=2a2adb769e55476b82e5014897e0a9c0)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:01:12.996687 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930a2c48-30df-4014-a064-369cfcb36056 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:01:13.024409 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82acd02-20dd-4250-a495-a654d0f8e7d7 admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:01:13.030517 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1174/2337] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 1 15:01:12 2022] PUT /identity/v3/projects/9962d041e4c84eee8a16348db69b92b9/users/0d74be5e44544f0b903451665c2122fb/roles/2a2adb769e55476b82e5014897e0a9c0 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 15:01:13.040071 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82acd02-20dd-4250-a495-a654d0f8e7d7 admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:01:13.192227 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7899bc-03ae-484e-a131-209230866b9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:01:13.192848 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7899bc-03ae-484e-a131-209230866b9c admin admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:01:13.193715 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7899bc-03ae-484e-a131-209230866b9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:01:13.193994 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7899bc-03ae-484e-a131-209230866b9c admin admin] PATH_INFO: `/v3/roles` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:01:13.194678 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7899bc-03ae-484e-a131-209230866b9c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:01:13.196311 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7899bc-03ae-484e-a131-209230866b9c admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:01:13.216594 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930a2c48-30df-4014-a064-369cfcb36056 admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:01:13.217638 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1164/2338] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 1 15:01:13 2022] GET /identity/v3/roles => generated 1420 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 15:01:13.232093 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930a2c48-30df-4014-a064-369cfcb36056 admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:01:13.241952 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c42a41-3214-4a1f-ad23-43f3936a49c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:01:13.242668 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c42a41-3214-4a1f-ad23-43f3936a49c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:01:13.242955 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c42a41-3214-4a1f-ad23-43f3936a49c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:01:13.243232 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c42a41-3214-4a1f-ad23-43f3936a49c6 admin admin] PATH_INFO: `/v3/projects/9962d041e4c84eee8a16348db69b92b9/users/0d74be5e44544f0b903451665c2122fb/roles/2a2adb769e55476b82e5014897e0a9c0` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:01:13.245933 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c42a41-3214-4a1f-ad23-43f3936a49c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9962d041e4c84eee8a16348db69b92b9, user_id=0d74be5e44544f0b903451665c2122fb, role_id=2a2adb769e55476b82e5014897e0a9c0)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:01:13.248764 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c42a41-3214-4a1f-ad23-43f3936a49c6 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:01:13.275507 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1175/2339] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 1 15:01:13 2022] PUT /identity/v3/projects/9962d041e4c84eee8a16348db69b92b9/users/0d74be5e44544f0b903451665c2122fb/roles/2a2adb769e55476b82e5014897e0a9c0 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 15:01:13.338092 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798c10e3-9bb6-4471-a9c4-262324167eda None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:01:13.338268 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798c10e3-9bb6-4471-a9c4-262324167eda None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:01:13.338564 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798c10e3-9bb6-4471-a9c4-262324167eda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:01:13.406683 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-798c10e3-9bb6-4471-a9c4-262324167eda None None] MFA Rules not processed for user `0d74be5e44544f0b903451665c2122fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 15:01:13.408039 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-798c10e3-9bb6-4471-a9c4-262324167eda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:01:13.523741 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1165/2340] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 15:01:13 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 15:01:13.535359 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c42a41-3214-4a1f-ad23-43f3936a49c6 admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:01:13.537209 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-a2c42a41-3214-4a1f-ad23-43f3936a49c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:01:13.670130 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c42a41-3214-4a1f-ad23-43f3936a49c6 admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:01:13.680369 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d071ad-5213-4420-9e51-d74ec08c7a98 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3b81739e34e4e648f88b6d73fcb5e46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:01:13.681001 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d071ad-5213-4420-9e51-d74ec08c7a98 service cinder] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:01:13.681315 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d071ad-5213-4420-9e51-d74ec08c7a98 service cinder] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:01:13.681590 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d071ad-5213-4420-9e51-d74ec08c7a98 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:01:13.708518 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d071ad-5213-4420-9e51-d74ec08c7a98 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:01:13.710223 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d071ad-5213-4420-9e51-d74ec08c7a98 service cinder] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:01:13.750185 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1176/2341] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 15:01:13 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 15:01:14.565959 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798c10e3-9bb6-4471-a9c4-262324167eda None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:01:14.575907 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798c10e3-9bb6-4471-a9c4-262324167eda None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:01:14.579910 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fe2b3f-e6c7-4665-b48f-c2f998ea76d5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3b81739e34e4e648f88b6d73fcb5e46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:01:14.580528 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fe2b3f-e6c7-4665-b48f-c2f998ea76d5 service cinder] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:01:14.580996 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fe2b3f-e6c7-4665-b48f-c2f998ea76d5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:01:14.581267 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fe2b3f-e6c7-4665-b48f-c2f998ea76d5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:01:14.593933 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 15:01:14.665504 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fe2b3f-e6c7-4665-b48f-c2f998ea76d5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:01:14.667415 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fe2b3f-e6c7-4665-b48f-c2f998ea76d5 service cinder] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:01:14.678670 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1166/2342] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 15:01:14 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 15:01:18.801503 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d071ad-5213-4420-9e51-d74ec08c7a98 service cinder] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:01:18.803437 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-83d071ad-5213-4420-9e51-d74ec08c7a98 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:01:18.815614 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d071ad-5213-4420-9e51-d74ec08c7a98 service cinder] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:01:18.824642 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e2341b-2639-4afe-9005-1a71433e7b74 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3b81739e34e4e648f88b6d73fcb5e46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:01:18.825266 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e2341b-2639-4afe-9005-1a71433e7b74 service cinder] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:01:18.825551 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e2341b-2639-4afe-9005-1a71433e7b74 service cinder] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:01:18.825825 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e2341b-2639-4afe-9005-1a71433e7b74 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:01:18.908080 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e2341b-2639-4afe-9005-1a71433e7b74 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:01:18.909563 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e2341b-2639-4afe-9005-1a71433e7b74 service cinder] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:01:18.921622 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1177/2343] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 15:01:18 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 15:01:19.175511 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590f5c1f-6ea6-4109-8a8f-2549d2ce694e None None] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:01:19.176266 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590f5c1f-6ea6-4109-8a8f-2549d2ce694e None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:01:19.176369 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590f5c1f-6ea6-4109-8a8f-2549d2ce694e None None] PATH_INFO: `/` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:01:19.180509 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1167/2344] 10.222.0.33 () {54 vars in 921 bytes} [Thu Sep 1 15:01:19 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 15:01:19.184880 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a39cebc-8bef-4d63-99b1-f4780c69f40c None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:01:19.185164 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a39cebc-8bef-4d63-99b1-f4780c69f40c None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:01:19.185639 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a39cebc-8bef-4d63-99b1-f4780c69f40c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:01:19.215461 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-8a39cebc-8bef-4d63-99b1-f4780c69f40c None None] MFA Rules not processed for user `4858bb8621854ec8aba836e7d9050498`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 15:01:19.229580 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-8a39cebc-8bef-4d63-99b1-f4780c69f40c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:01:19.327435 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 15:01:19.344079 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1178/2345] 10.222.0.33 () {58 vars in 1018 bytes} [Thu Sep 1 15:01:19 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 15:01:19.363699 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590f5c1f-6ea6-4109-8a8f-2549d2ce694e None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:01:19.366908 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-590f5c1f-6ea6-4109-8a8f-2549d2ce694e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:01:19.463851 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590f5c1f-6ea6-4109-8a8f-2549d2ce694e None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:01:19.473821 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843914ba-f329-4bc1-b082-1a4978563b79 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3069f72ce98049b99c9d98229a62bb51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:01:19.475199 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843914ba-f329-4bc1-b082-1a4978563b79 service glance] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:01:19.475442 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843914ba-f329-4bc1-b082-1a4978563b79 service glance] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:01:19.476194 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843914ba-f329-4bc1-b082-1a4978563b79 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:01:19.490157 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843914ba-f329-4bc1-b082-1a4978563b79 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:01:19.492372 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843914ba-f329-4bc1-b082-1a4978563b79 service glance] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:01:19.511391 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1168/2346] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 15:01:19 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 15:01:19.590688 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cada9b-c15c-4b40-83df-777b2a002729 None None] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:01:19.591039 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cada9b-c15c-4b40-83df-777b2a002729 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:01:19.592293 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cada9b-c15c-4b40-83df-777b2a002729 None None] PATH_INFO: `/` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:01:19.593670 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1179/2347] 10.222.0.33 () {54 vars in 921 bytes} [Thu Sep 1 15:01:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 15:01:19.599005 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effb018a-ae3c-4aa2-bfb6-8ce0ab666cad None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:01:19.599292 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effb018a-ae3c-4aa2-bfb6-8ce0ab666cad None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:01:19.599557 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effb018a-ae3c-4aa2-bfb6-8ce0ab666cad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:01:19.620412 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-effb018a-ae3c-4aa2-bfb6-8ce0ab666cad None None] MFA Rules not processed for user `2790149639aa418bbc4e4982e93eda44`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 15:01:19.621623 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-effb018a-ae3c-4aa2-bfb6-8ce0ab666cad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:01:19.691490 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1169/2348] 10.222.0.33 () {58 vars in 1018 bytes} [Thu Sep 1 15:01:19 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 15:01:19.705517 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cada9b-c15c-4b40-83df-777b2a002729 None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:01:19.708250 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-22cada9b-c15c-4b40-83df-777b2a002729 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:01:19.723000 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cada9b-c15c-4b40-83df-777b2a002729 None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:01:19.727253 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4960ef3a-e546-4167-a463-09c1e3ff5200 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2790149639aa418bbc4e4982e93eda44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:01:19.727853 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4960ef3a-e546-4167-a463-09c1e3ff5200 service neutron] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:01:19.728125 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4960ef3a-e546-4167-a463-09c1e3ff5200 service neutron] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:01:19.728385 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4960ef3a-e546-4167-a463-09c1e3ff5200 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:01:19.735855 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4960ef3a-e546-4167-a463-09c1e3ff5200 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:01:19.737271 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4960ef3a-e546-4167-a463-09c1e3ff5200 service neutron] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:01:19.745307 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1180/2349] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 15:01:19 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 15:01:19.839857 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7667684-a3a9-4380-a45f-03ada6144c56 None None] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:01:19.840147 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7667684-a3a9-4380-a45f-03ada6144c56 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:01:19.840450 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7667684-a3a9-4380-a45f-03ada6144c56 None None] PATH_INFO: `/` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:01:19.841791 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1170/2350] 10.222.0.33 () {54 vars in 921 bytes} [Thu Sep 1 15:01:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 15:01:19.847259 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98533d2e-57ee-408e-8ec5-5f32104df292 None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:01:19.847735 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98533d2e-57ee-408e-8ec5-5f32104df292 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:01:19.847851 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98533d2e-57ee-408e-8ec5-5f32104df292 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:01:19.867227 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-98533d2e-57ee-408e-8ec5-5f32104df292 None None] MFA Rules not processed for user `4858bb8621854ec8aba836e7d9050498`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 15:01:19.868426 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-98533d2e-57ee-408e-8ec5-5f32104df292 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:01:19.878651 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1181/2351] 10.222.0.33 () {58 vars in 1018 bytes} [Thu Sep 1 15:01:19 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 15:01:19.889609 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7667684-a3a9-4380-a45f-03ada6144c56 None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:01:19.891559 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-c7667684-a3a9-4380-a45f-03ada6144c56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:01:19.905653 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7667684-a3a9-4380-a45f-03ada6144c56 None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:01:19.909260 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbdd15a-8516-427e-9416-8107f4bd887b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2790149639aa418bbc4e4982e93eda44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:01:19.909855 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbdd15a-8516-427e-9416-8107f4bd887b service neutron] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:01:19.910134 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbdd15a-8516-427e-9416-8107f4bd887b service neutron] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:01:19.910432 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbdd15a-8516-427e-9416-8107f4bd887b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:01:19.920355 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbdd15a-8516-427e-9416-8107f4bd887b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:01:19.921790 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbdd15a-8516-427e-9416-8107f4bd887b service neutron] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:01:19.932565 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1171/2352] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 15:01:19 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 15:01:21.289279 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98533d2e-57ee-408e-8ec5-5f32104df292 None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:01:21.358593 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98533d2e-57ee-408e-8ec5-5f32104df292 None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:01:21.363819 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406efd9c-5116-48cd-af33-86893dd84711 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:01:21.364531 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406efd9c-5116-48cd-af33-86893dd84711 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:01:21.364874 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406efd9c-5116-48cd-af33-86893dd84711 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:01:21.365208 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406efd9c-5116-48cd-af33-86893dd84711 admin admin] PATH_INFO: `/v3/users/d869e6cc78424763bad6a2bb76996f85` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:01:21.367028 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406efd9c-5116-48cd-af33-86893dd84711 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d869e6cc78424763bad6a2bb76996f85)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:01:21.369150 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406efd9c-5116-48cd-af33-86893dd84711 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:01:21.435349 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-406efd9c-5116-48cd-af33-86893dd84711 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd869e6cc78424763bad6a2bb76996f85'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:01:21.444497 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-406efd9c-5116-48cd-af33-86893dd84711 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd869e6cc78424763bad6a2bb76996f85'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:01:21.450828 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-406efd9c-5116-48cd-af33-86893dd84711 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd869e6cc78424763bad6a2bb76996f85'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:01:21.466839 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-406efd9c-5116-48cd-af33-86893dd84711 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd869e6cc78424763bad6a2bb76996f85'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:01:21.467423 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-406efd9c-5116-48cd-af33-86893dd84711 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd869e6cc78424763bad6a2bb76996f85'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:01:21.483084 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1182/2353] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Sep 1 15:01:21 2022] DELETE /identity/v3/users/d869e6cc78424763bad6a2bb76996f85 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 15:01:21.487951 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97177317-9529-49d0-9782-5c57ae9ea76e None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:01:21.488467 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97177317-9529-49d0-9782-5c57ae9ea76e None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:01:21.488572 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97177317-9529-49d0-9782-5c57ae9ea76e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:01:21.511267 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-97177317-9529-49d0-9782-5c57ae9ea76e None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 15:01:21.512667 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-97177317-9529-49d0-9782-5c57ae9ea76e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:01:21.563493 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 15:01:21.583757 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1172/2354] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 15:01:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 15:01:21.599575 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406efd9c-5116-48cd-af33-86893dd84711 admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:01:21.601869 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-406efd9c-5116-48cd-af33-86893dd84711 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:01:21.702066 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406efd9c-5116-48cd-af33-86893dd84711 admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:01:21.712543 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92267a46-cee7-48c0-b4e9-f1d8febdd7d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2790149639aa418bbc4e4982e93eda44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:01:21.713699 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92267a46-cee7-48c0-b4e9-f1d8febdd7d1 service neutron] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:01:21.714180 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92267a46-cee7-48c0-b4e9-f1d8febdd7d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:01:21.714696 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92267a46-cee7-48c0-b4e9-f1d8febdd7d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:01:21.726465 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92267a46-cee7-48c0-b4e9-f1d8febdd7d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:01:21.728159 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92267a46-cee7-48c0-b4e9-f1d8febdd7d1 service neutron] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:01:21.734850 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 15:01:21.743750 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1183/2355] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 15:01:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 15:01:22.059829 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97177317-9529-49d0-9782-5c57ae9ea76e None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:01:22.061321 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-97177317-9529-49d0-9782-5c57ae9ea76e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:01:22.074554 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97177317-9529-49d0-9782-5c57ae9ea76e None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:01:22.079111 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddf7212-381c-442b-831d-5ec5e473f101 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:01:22.079728 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddf7212-381c-442b-831d-5ec5e473f101 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:01:22.080015 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddf7212-381c-442b-831d-5ec5e473f101 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:01:22.080287 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddf7212-381c-442b-831d-5ec5e473f101 admin admin] PATH_INFO: `/v3/projects/42f317e30f0945928fabbcad65551c87` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:01:22.081414 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bddf7212-381c-442b-831d-5ec5e473f101 admin admin] RBAC: Authorizing `identity:delete_project(project_id=42f317e30f0945928fabbcad65551c87)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:01:22.083462 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bddf7212-381c-442b-831d-5ec5e473f101 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:01:22.135750 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-bddf7212-381c-442b-831d-5ec5e473f101 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '42f317e30f0945928fabbcad65551c87', 'request_id': 'req-bddf7212-381c-442b-831d-5ec5e473f101'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:01:22.142386 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-bddf7212-381c-442b-831d-5ec5e473f101 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '42f317e30f0945928fabbcad65551c87', 'request_id': 'req-bddf7212-381c-442b-831d-5ec5e473f101'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:01:22.150061 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-bddf7212-381c-442b-831d-5ec5e473f101 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '42f317e30f0945928fabbcad65551c87', 'request_id': 'req-bddf7212-381c-442b-831d-5ec5e473f101'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:01:22.164017 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-bddf7212-381c-442b-831d-5ec5e473f101 admin admin] The token cache is being invalidate because project 42f317e30f0945928fabbcad65551c87 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81341) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 15:01:22.164310 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-bddf7212-381c-442b-831d-5ec5e473f101 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:01:22.165215 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-bddf7212-381c-442b-831d-5ec5e473f101 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:01:22.166781 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1173/2356] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Sep 1 15:01:22 2022] DELETE /identity/v3/projects/42f317e30f0945928fabbcad65551c87 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 15:01:22.170845 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92267a46-cee7-48c0-b4e9-f1d8febdd7d1 service neutron] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:01:22.181469 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92267a46-cee7-48c0-b4e9-f1d8febdd7d1 service neutron] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:01:22.246259 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4e76e6-8af7-47e5-8fd7-37fcd4e97cc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:01:22.247055 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4e76e6-8af7-47e5-8fd7-37fcd4e97cc2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:01:22.247330 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4e76e6-8af7-47e5-8fd7-37fcd4e97cc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:01:22.247596 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4e76e6-8af7-47e5-8fd7-37fcd4e97cc2 admin admin] PATH_INFO: `/v3/users/0ee79dda1c8c4ec8ad0e38a5f8b296c1` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:01:22.248754 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4e76e6-8af7-47e5-8fd7-37fcd4e97cc2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0ee79dda1c8c4ec8ad0e38a5f8b296c1)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:01:22.250773 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4e76e6-8af7-47e5-8fd7-37fcd4e97cc2 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:01:22.314509 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-4b4e76e6-8af7-47e5-8fd7-37fcd4e97cc2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0ee79dda1c8c4ec8ad0e38a5f8b296c1'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:01:22.323068 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-4b4e76e6-8af7-47e5-8fd7-37fcd4e97cc2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0ee79dda1c8c4ec8ad0e38a5f8b296c1'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:01:22.323718 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-4b4e76e6-8af7-47e5-8fd7-37fcd4e97cc2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0ee79dda1c8c4ec8ad0e38a5f8b296c1'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:01:22.339474 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-4b4e76e6-8af7-47e5-8fd7-37fcd4e97cc2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0ee79dda1c8c4ec8ad0e38a5f8b296c1'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:01:22.340210 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-4b4e76e6-8af7-47e5-8fd7-37fcd4e97cc2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0ee79dda1c8c4ec8ad0e38a5f8b296c1'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:01:22.356322 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1184/2357] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Sep 1 15:01:22 2022] DELETE /identity/v3/users/0ee79dda1c8c4ec8ad0e38a5f8b296c1 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 15:01:22.655451 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddf7212-381c-442b-831d-5ec5e473f101 admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:01:22.657989 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-bddf7212-381c-442b-831d-5ec5e473f101 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:01:22.682181 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddf7212-381c-442b-831d-5ec5e473f101 admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:01:22.747508 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ac0652-6cce-4ac7-9246-765c2e61809f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:01:22.748146 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ac0652-6cce-4ac7-9246-765c2e61809f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:01:22.748424 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ac0652-6cce-4ac7-9246-765c2e61809f admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:01:22.748716 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ac0652-6cce-4ac7-9246-765c2e61809f admin admin] PATH_INFO: `/v3/projects/9595ebf6550f43eb80b0a716c466644e` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:01:22.749897 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ac0652-6cce-4ac7-9246-765c2e61809f admin admin] RBAC: Authorizing `identity:delete_project(project_id=9595ebf6550f43eb80b0a716c466644e)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:01:22.751790 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ac0652-6cce-4ac7-9246-765c2e61809f admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:01:22.809268 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-38ac0652-6cce-4ac7-9246-765c2e61809f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9595ebf6550f43eb80b0a716c466644e', 'request_id': 'req-38ac0652-6cce-4ac7-9246-765c2e61809f'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:01:22.821144 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-38ac0652-6cce-4ac7-9246-765c2e61809f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9595ebf6550f43eb80b0a716c466644e', 'request_id': 'req-38ac0652-6cce-4ac7-9246-765c2e61809f'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:01:22.841705 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-38ac0652-6cce-4ac7-9246-765c2e61809f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9595ebf6550f43eb80b0a716c466644e', 'request_id': 'req-38ac0652-6cce-4ac7-9246-765c2e61809f'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:01:22.879432 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-38ac0652-6cce-4ac7-9246-765c2e61809f admin admin] The token cache is being invalidate because project 9595ebf6550f43eb80b0a716c466644e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81341) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 15:01:22.879928 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-38ac0652-6cce-4ac7-9246-765c2e61809f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:01:22.881701 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-38ac0652-6cce-4ac7-9246-765c2e61809f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:01:22.889046 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1174/2358] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Sep 1 15:01:22 2022] DELETE /identity/v3/projects/9595ebf6550f43eb80b0a716c466644e => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 15:01:22.892914 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4e76e6-8af7-47e5-8fd7-37fcd4e97cc2 admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:01:22.895098 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-4b4e76e6-8af7-47e5-8fd7-37fcd4e97cc2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:01:22.920652 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4e76e6-8af7-47e5-8fd7-37fcd4e97cc2 admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:01:23.035271 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9352ef64-6d88-40ab-a22f-d42513acb8c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:01:23.035967 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9352ef64-6d88-40ab-a22f-d42513acb8c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:01:23.036248 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9352ef64-6d88-40ab-a22f-d42513acb8c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:01:23.036514 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9352ef64-6d88-40ab-a22f-d42513acb8c9 admin admin] PATH_INFO: `/v3/users/0d74be5e44544f0b903451665c2122fb` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:01:23.038399 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9352ef64-6d88-40ab-a22f-d42513acb8c9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0d74be5e44544f0b903451665c2122fb)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:01:23.040467 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9352ef64-6d88-40ab-a22f-d42513acb8c9 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:01:23.076942 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 15:01:23.119280 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-9352ef64-6d88-40ab-a22f-d42513acb8c9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0d74be5e44544f0b903451665c2122fb'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:01:23.128976 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-9352ef64-6d88-40ab-a22f-d42513acb8c9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0d74be5e44544f0b903451665c2122fb'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:01:23.129718 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-9352ef64-6d88-40ab-a22f-d42513acb8c9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0d74be5e44544f0b903451665c2122fb'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:01:23.143427 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-9352ef64-6d88-40ab-a22f-d42513acb8c9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0d74be5e44544f0b903451665c2122fb'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:01:23.144100 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-9352ef64-6d88-40ab-a22f-d42513acb8c9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0d74be5e44544f0b903451665c2122fb'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:01:23.158361 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1185/2359] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Sep 1 15:01:22 2022] DELETE /identity/v3/users/0d74be5e44544f0b903451665c2122fb => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 15:01:23.247155 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad3e6fa-bc87-4b35-9f52-05b8d8559e35 None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:01:23.247502 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad3e6fa-bc87-4b35-9f52-05b8d8559e35 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:01:23.247870 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad3e6fa-bc87-4b35-9f52-05b8d8559e35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:01:23.256823 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 15:01:23.269409 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-1ad3e6fa-bc87-4b35-9f52-05b8d8559e35 None None] MFA Rules not processed for user `24b6f275ffb04ffeae3c2097bfd040f5`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 15:01:23.271218 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-1ad3e6fa-bc87-4b35-9f52-05b8d8559e35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:01:23.336327 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1175/2360] 10.222.0.33 () {58 vars in 1009 bytes} [Thu Sep 1 15:01:23 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 15:01:23.349318 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9352ef64-6d88-40ab-a22f-d42513acb8c9 admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:01:23.351393 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-9352ef64-6d88-40ab-a22f-d42513acb8c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:01:23.422832 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9352ef64-6d88-40ab-a22f-d42513acb8c9 admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:01:23.428128 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e4e3e2-13e3-4c86-bc45-1c629036f10c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73a5bda5a6ea46e9863ccafd57dee339', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:01:23.428738 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e4e3e2-13e3-4c86-bc45-1c629036f10c service swift] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:01:23.429018 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e4e3e2-13e3-4c86-bc45-1c629036f10c service swift] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:01:23.429289 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e4e3e2-13e3-4c86-bc45-1c629036f10c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:01:23.437520 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e4e3e2-13e3-4c86-bc45-1c629036f10c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:01:23.439149 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e4e3e2-13e3-4c86-bc45-1c629036f10c service swift] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:01:23.451974 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1186/2361] 10.222.0.33 () {60 vars in 1384 bytes} [Thu Sep 1 15:01:23 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 01 15:01:23.452276 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad3e6fa-bc87-4b35-9f52-05b8d8559e35 None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:01:23.457697 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-1ad3e6fa-bc87-4b35-9f52-05b8d8559e35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:01:23.592301 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad3e6fa-bc87-4b35-9f52-05b8d8559e35 None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:01:23.603133 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13210c00-e917-4e35-9594-6e89ffe665e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:01:23.604697 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13210c00-e917-4e35-9594-6e89ffe665e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:01:23.605155 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13210c00-e917-4e35-9594-6e89ffe665e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:01:23.605610 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13210c00-e917-4e35-9594-6e89ffe665e0 admin admin] PATH_INFO: `/v3/projects/9962d041e4c84eee8a16348db69b92b9` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:01:23.607882 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13210c00-e917-4e35-9594-6e89ffe665e0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9962d041e4c84eee8a16348db69b92b9)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:01:23.612184 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13210c00-e917-4e35-9594-6e89ffe665e0 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:01:23.689724 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-13210c00-e917-4e35-9594-6e89ffe665e0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9962d041e4c84eee8a16348db69b92b9', 'request_id': 'req-13210c00-e917-4e35-9594-6e89ffe665e0'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:01:23.696544 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-13210c00-e917-4e35-9594-6e89ffe665e0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9962d041e4c84eee8a16348db69b92b9', 'request_id': 'req-13210c00-e917-4e35-9594-6e89ffe665e0'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:01:23.708559 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-13210c00-e917-4e35-9594-6e89ffe665e0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9962d041e4c84eee8a16348db69b92b9', 'request_id': 'req-13210c00-e917-4e35-9594-6e89ffe665e0'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:01:23.739350 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-13210c00-e917-4e35-9594-6e89ffe665e0 admin admin] The token cache is being invalidate because project 9962d041e4c84eee8a16348db69b92b9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81341) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 15:01:23.740285 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-13210c00-e917-4e35-9594-6e89ffe665e0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:01:23.741680 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-13210c00-e917-4e35-9594-6e89ffe665e0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:01:23.746967 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1176/2362] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Sep 1 15:01:23 2022] DELETE /identity/v3/projects/9962d041e4c84eee8a16348db69b92b9 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 15:01:23.755585 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcf3e98-bf1e-44ed-a7ca-2faf4a358171 None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:01:23.755585 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcf3e98-bf1e-44ed-a7ca-2faf4a358171 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:01:23.755585 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcf3e98-bf1e-44ed-a7ca-2faf4a358171 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:01:23.792276 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-9fcf3e98-bf1e-44ed-a7ca-2faf4a358171 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 15:01:23.793654 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-9fcf3e98-bf1e-44ed-a7ca-2faf4a358171 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:01:23.900239 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1187/2363] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 15:01:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 15:01:23.908763 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e49509-c537-42ec-adff-06c78213a385 None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:01:23.909302 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e49509-c537-42ec-adff-06c78213a385 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:01:23.909818 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e49509-c537-42ec-adff-06c78213a385 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:01:23.931621 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-81e49509-c537-42ec-adff-06c78213a385 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 15:01:23.935676 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-81e49509-c537-42ec-adff-06c78213a385 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:01:23.944417 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1177/2364] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 15:01:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 15:01:23.949079 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcf3e98-bf1e-44ed-a7ca-2faf4a358171 None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:01:23.962215 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcf3e98-bf1e-44ed-a7ca-2faf4a358171 None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:01:23.966936 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2553ad78-f818-4d79-99d0-9727b2c0a1c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:01:23.967597 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2553ad78-f818-4d79-99d0-9727b2c0a1c2 admin admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:01:23.967906 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2553ad78-f818-4d79-99d0-9727b2c0a1c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:01:23.968173 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2553ad78-f818-4d79-99d0-9727b2c0a1c2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:01:23.968838 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2553ad78-f818-4d79-99d0-9727b2c0a1c2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 15:01:23.969108 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2553ad78-f818-4d79-99d0-9727b2c0a1c2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:01:23.970972 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2553ad78-f818-4d79-99d0-9727b2c0a1c2 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:01:23.984533 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1188/2365] 10.222.0.33 () {58 vars in 1153 bytes} [Thu Sep 1 15:01:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 15:01:23.989121 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e49509-c537-42ec-adff-06c78213a385 None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:01:23.999224 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e49509-c537-42ec-adff-06c78213a385 None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:01:24.004143 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7baf4ba8-8397-47c3-a7a1-35b6a0e405d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:01:24.004743 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baf4ba8-8397-47c3-a7a1-35b6a0e405d5 admin admin] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:01:24.005019 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baf4ba8-8397-47c3-a7a1-35b6a0e405d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:01:24.005519 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baf4ba8-8397-47c3-a7a1-35b6a0e405d5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:01:24.006321 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7baf4ba8-8397-47c3-a7a1-35b6a0e405d5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:01:24.008750 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7baf4ba8-8397-47c3-a7a1-35b6a0e405d5 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:01:24.053094 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1178/2366] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Sep 1 15:01:23 2022] POST /identity/v3/projects => generated 383 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 15:01:24.054424 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2553ad78-f818-4d79-99d0-9727b2c0a1c2 admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:01:24.065527 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2553ad78-f818-4d79-99d0-9727b2c0a1c2 admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:01:24.075823 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 15:01:24.153689 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcc191c-92a8-47bf-bebf-54435b37e99d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:01:24.154418 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcc191c-92a8-47bf-bebf-54435b37e99d admin admin] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:01:24.154771 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcc191c-92a8-47bf-bebf-54435b37e99d admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:01:24.155045 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcc191c-92a8-47bf-bebf-54435b37e99d admin admin] PATH_INFO: `/v3/users` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:01:24.155761 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcc191c-92a8-47bf-bebf-54435b37e99d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:01:24.157984 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcc191c-92a8-47bf-bebf-54435b37e99d admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:01:24.223984 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1189/2367] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Sep 1 15:01:24 2022] POST /identity/v3/users => generated 292 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 15:01:24.228339 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7baf4ba8-8397-47c3-a7a1-35b6a0e405d5 admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:01:24.244857 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7baf4ba8-8397-47c3-a7a1-35b6a0e405d5 admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:01:24.259232 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc64492c-c7e6-4bd8-bd50-07aeb81afb03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:01:24.259762 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc64492c-c7e6-4bd8-bd50-07aeb81afb03 admin admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:01:24.259910 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc64492c-c7e6-4bd8-bd50-07aeb81afb03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:01:24.260480 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc64492c-c7e6-4bd8-bd50-07aeb81afb03 admin admin] PATH_INFO: `/v3/roles` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:01:24.260801 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc64492c-c7e6-4bd8-bd50-07aeb81afb03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:01:24.266614 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc64492c-c7e6-4bd8-bd50-07aeb81afb03 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:01:24.273469 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 15:01:24.289072 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1179/2368] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 1 15:01:24 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 15:01:24.290582 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcc191c-92a8-47bf-bebf-54435b37e99d admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:01:24.314225 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcc191c-92a8-47bf-bebf-54435b37e99d admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:01:24.320573 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e89da4-d7cb-4aa0-8a2a-5cc05c822281 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:01:24.321553 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e89da4-d7cb-4aa0-8a2a-5cc05c822281 admin admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:01:24.321694 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e89da4-d7cb-4aa0-8a2a-5cc05c822281 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:01:24.322028 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e89da4-d7cb-4aa0-8a2a-5cc05c822281 admin admin] PATH_INFO: `/v3/roles` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:01:24.322777 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e89da4-d7cb-4aa0-8a2a-5cc05c822281 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:01:24.324661 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e89da4-d7cb-4aa0-8a2a-5cc05c822281 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:01:24.348269 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1190/2369] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 1 15:01:24 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 15:01:24.354616 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc64492c-c7e6-4bd8-bd50-07aeb81afb03 admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:01:24.375753 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc64492c-c7e6-4bd8-bd50-07aeb81afb03 admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:01:24.391934 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0acea1b-ab81-4e60-98ce-95903b7f2e7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:01:24.392619 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0acea1b-ab81-4e60-98ce-95903b7f2e7b admin admin] REQUEST_METHOD: `PUT` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:01:24.392903 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0acea1b-ab81-4e60-98ce-95903b7f2e7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:01:24.393186 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0acea1b-ab81-4e60-98ce-95903b7f2e7b admin admin] PATH_INFO: `/v3/projects/a1a751ba0d92407a80cece63c25b1f24/users/eb0c7bf504fd42c1b1a3e34b3e365060/roles/5d1feef0e13d4874928bd86258c4e3b2` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:01:24.462862 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0acea1b-ab81-4e60-98ce-95903b7f2e7b admin admin] RBAC: Authorizing `identity:create_grant(project_id=a1a751ba0d92407a80cece63c25b1f24, user_id=eb0c7bf504fd42c1b1a3e34b3e365060, role_id=5d1feef0e13d4874928bd86258c4e3b2)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:01:24.466070 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0acea1b-ab81-4e60-98ce-95903b7f2e7b admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:01:24.485196 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1180/2370] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 1 15:01:24 2022] PUT /identity/v3/projects/a1a751ba0d92407a80cece63c25b1f24/users/eb0c7bf504fd42c1b1a3e34b3e365060/roles/5d1feef0e13d4874928bd86258c4e3b2 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 15:01:24.502328 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6acc2a-86e3-4065-84ba-88517f1de46a None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:01:24.502852 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6acc2a-86e3-4065-84ba-88517f1de46a None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:01:24.503097 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6acc2a-86e3-4065-84ba-88517f1de46a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:01:24.579297 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-ce6acc2a-86e3-4065-84ba-88517f1de46a None None] MFA Rules not processed for user `eb0c7bf504fd42c1b1a3e34b3e365060`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 15:01:24.581143 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-ce6acc2a-86e3-4065-84ba-88517f1de46a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:01:24.679967 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1191/2371] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 15:01:24 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 15:01:24.684468 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0acea1b-ab81-4e60-98ce-95903b7f2e7b admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:01:24.762231 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0acea1b-ab81-4e60-98ce-95903b7f2e7b admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:01:24.767252 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a471d6d-26bf-4a62-915a-aa7489cd4fd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:01:24.767864 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a471d6d-26bf-4a62-915a-aa7489cd4fd8 admin admin] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:01:24.768147 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a471d6d-26bf-4a62-915a-aa7489cd4fd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:01:24.768425 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a471d6d-26bf-4a62-915a-aa7489cd4fd8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:01:24.769072 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a471d6d-26bf-4a62-915a-aa7489cd4fd8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:01:24.771087 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a471d6d-26bf-4a62-915a-aa7489cd4fd8 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:01:24.812424 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1181/2372] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Sep 1 15:01:24 2022] POST /identity/v3/projects => generated 381 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 15:01:24.817767 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6acc2a-86e3-4065-84ba-88517f1de46a None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:01:24.844130 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6acc2a-86e3-4065-84ba-88517f1de46a None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:01:24.935503 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339855b2-ebf8-4bc5-b20e-ecfde1f1f68b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:01:24.936796 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339855b2-ebf8-4bc5-b20e-ecfde1f1f68b admin admin] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:01:24.937361 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339855b2-ebf8-4bc5-b20e-ecfde1f1f68b admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:01:24.938345 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339855b2-ebf8-4bc5-b20e-ecfde1f1f68b admin admin] PATH_INFO: `/v3/users` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:01:24.939524 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339855b2-ebf8-4bc5-b20e-ecfde1f1f68b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:01:24.941967 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339855b2-ebf8-4bc5-b20e-ecfde1f1f68b admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:01:24.967658 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1192/2373] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Sep 1 15:01:24 2022] POST /identity/v3/users => generated 297 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 15:01:24.971537 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a471d6d-26bf-4a62-915a-aa7489cd4fd8 admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:01:24.982704 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a471d6d-26bf-4a62-915a-aa7489cd4fd8 admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:01:24.987441 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4b9034-2e5a-4444-a6fa-cf386df3f069 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:01:24.988052 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4b9034-2e5a-4444-a6fa-cf386df3f069 admin admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:01:24.988325 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4b9034-2e5a-4444-a6fa-cf386df3f069 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:01:24.988601 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4b9034-2e5a-4444-a6fa-cf386df3f069 admin admin] PATH_INFO: `/v3/roles` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:01:24.989243 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4b9034-2e5a-4444-a6fa-cf386df3f069 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:01:24.991034 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4b9034-2e5a-4444-a6fa-cf386df3f069 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:01:25.006336 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1182/2374] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 1 15:01:24 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 15:01:25.007867 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339855b2-ebf8-4bc5-b20e-ecfde1f1f68b admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:01:25.020572 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339855b2-ebf8-4bc5-b20e-ecfde1f1f68b admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:01:25.025980 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6af350a-2adc-4aef-a204-e6b01b7f2894 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:01:25.026723 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6af350a-2adc-4aef-a204-e6b01b7f2894 admin admin] REQUEST_METHOD: `PUT` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:01:25.027179 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6af350a-2adc-4aef-a204-e6b01b7f2894 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:01:25.027463 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6af350a-2adc-4aef-a204-e6b01b7f2894 admin admin] PATH_INFO: `/v3/projects/c2e8733b7dd847efa524c35ffb316deb/users/c5c6e682a4ef4f33aeb24f03c712370f/roles/2a2adb769e55476b82e5014897e0a9c0` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:01:25.036722 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 15:01:25.045032 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6af350a-2adc-4aef-a204-e6b01b7f2894 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c2e8733b7dd847efa524c35ffb316deb, user_id=c5c6e682a4ef4f33aeb24f03c712370f, role_id=2a2adb769e55476b82e5014897e0a9c0)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:01:25.048215 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6af350a-2adc-4aef-a204-e6b01b7f2894 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:01:25.064276 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1193/2375] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 1 15:01:25 2022] PUT /identity/v3/projects/c2e8733b7dd847efa524c35ffb316deb/users/c5c6e682a4ef4f33aeb24f03c712370f/roles/2a2adb769e55476b82e5014897e0a9c0 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 15:01:25.068903 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4b9034-2e5a-4444-a6fa-cf386df3f069 admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:01:25.080544 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4b9034-2e5a-4444-a6fa-cf386df3f069 admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:01:25.146485 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f1213b-f3f8-4628-b5dc-8619a59a5042 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:01:25.148649 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f1213b-f3f8-4628-b5dc-8619a59a5042 admin admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:01:25.148929 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f1213b-f3f8-4628-b5dc-8619a59a5042 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:01:25.149202 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f1213b-f3f8-4628-b5dc-8619a59a5042 admin admin] PATH_INFO: `/v3/roles` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:01:25.149853 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f1213b-f3f8-4628-b5dc-8619a59a5042 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:01:25.151702 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f1213b-f3f8-4628-b5dc-8619a59a5042 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:01:25.164538 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1183/2376] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 1 15:01:25 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 15:01:25.168504 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6af350a-2adc-4aef-a204-e6b01b7f2894 admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:01:25.182723 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6af350a-2adc-4aef-a204-e6b01b7f2894 admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:01:25.188575 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1869080e-91a0-4054-8966-f91bc702a904 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:01:25.189246 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1869080e-91a0-4054-8966-f91bc702a904 admin admin] REQUEST_METHOD: `PUT` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:01:25.189528 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1869080e-91a0-4054-8966-f91bc702a904 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:01:25.189799 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1869080e-91a0-4054-8966-f91bc702a904 admin admin] PATH_INFO: `/v3/projects/c2e8733b7dd847efa524c35ffb316deb/users/c5c6e682a4ef4f33aeb24f03c712370f/roles/2a2adb769e55476b82e5014897e0a9c0` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:01:25.191781 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1869080e-91a0-4054-8966-f91bc702a904 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c2e8733b7dd847efa524c35ffb316deb, user_id=c5c6e682a4ef4f33aeb24f03c712370f, role_id=2a2adb769e55476b82e5014897e0a9c0)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:01:25.195068 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1869080e-91a0-4054-8966-f91bc702a904 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:01:25.207115 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1194/2377] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 1 15:01:25 2022] PUT /identity/v3/projects/c2e8733b7dd847efa524c35ffb316deb/users/c5c6e682a4ef4f33aeb24f03c712370f/roles/2a2adb769e55476b82e5014897e0a9c0 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 15:01:25.217879 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0d4336-442f-4d37-95fe-b20ce9d1c296 None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:01:25.218170 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0d4336-442f-4d37-95fe-b20ce9d1c296 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:01:25.218457 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0d4336-442f-4d37-95fe-b20ce9d1c296 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:01:25.247062 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-cf0d4336-442f-4d37-95fe-b20ce9d1c296 None None] MFA Rules not processed for user `c5c6e682a4ef4f33aeb24f03c712370f`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 15:01:25.248324 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-cf0d4336-442f-4d37-95fe-b20ce9d1c296 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:01:25.286560 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 15:01:25.326923 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1184/2378] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 15:01:25 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 15:01:25.340696 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1869080e-91a0-4054-8966-f91bc702a904 admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:01:25.342586 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-1869080e-91a0-4054-8966-f91bc702a904 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:01:25.426168 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1869080e-91a0-4054-8966-f91bc702a904 admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:01:25.430616 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca7b69b-568b-4689-9017-5210731c046e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3b81739e34e4e648f88b6d73fcb5e46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:01:25.431231 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca7b69b-568b-4689-9017-5210731c046e service cinder] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:01:25.431516 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca7b69b-568b-4689-9017-5210731c046e service cinder] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:01:25.431791 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca7b69b-568b-4689-9017-5210731c046e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:01:25.441631 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca7b69b-568b-4689-9017-5210731c046e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:01:25.443091 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca7b69b-568b-4689-9017-5210731c046e service cinder] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:01:25.454371 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1195/2379] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 15:01:25 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 15:01:25.603852 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0d4336-442f-4d37-95fe-b20ce9d1c296 None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:01:25.620584 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0d4336-442f-4d37-95fe-b20ce9d1c296 None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:01:25.634432 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce150a5-24d8-47e5-9acf-6c4a8303eb04 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3b81739e34e4e648f88b6d73fcb5e46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:01:25.635066 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce150a5-24d8-47e5-9acf-6c4a8303eb04 service cinder] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:01:25.635343 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce150a5-24d8-47e5-9acf-6c4a8303eb04 service cinder] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:01:25.635615 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce150a5-24d8-47e5-9acf-6c4a8303eb04 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:01:25.738721 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce150a5-24d8-47e5-9acf-6c4a8303eb04 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:01:25.740803 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce150a5-24d8-47e5-9acf-6c4a8303eb04 service cinder] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:01:25.762663 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1185/2380] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 15:01:25 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 15:01:28.460973 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca7b69b-568b-4689-9017-5210731c046e service cinder] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:01:28.524029 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca7b69b-568b-4689-9017-5210731c046e service cinder] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:01:28.528522 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa08a93d-0b27-4036-bbdf-f8374aba68d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:01:28.529154 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa08a93d-0b27-4036-bbdf-f8374aba68d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:01:28.529431 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa08a93d-0b27-4036-bbdf-f8374aba68d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:01:28.529700 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa08a93d-0b27-4036-bbdf-f8374aba68d1 admin admin] PATH_INFO: `/v3/users/eb0c7bf504fd42c1b1a3e34b3e365060` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:01:28.530846 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa08a93d-0b27-4036-bbdf-f8374aba68d1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eb0c7bf504fd42c1b1a3e34b3e365060)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:01:28.532763 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa08a93d-0b27-4036-bbdf-f8374aba68d1 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:01:28.580652 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-aa08a93d-0b27-4036-bbdf-f8374aba68d1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eb0c7bf504fd42c1b1a3e34b3e365060'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:01:28.588942 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-aa08a93d-0b27-4036-bbdf-f8374aba68d1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eb0c7bf504fd42c1b1a3e34b3e365060'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:01:28.589593 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-aa08a93d-0b27-4036-bbdf-f8374aba68d1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eb0c7bf504fd42c1b1a3e34b3e365060'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:01:28.606621 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-aa08a93d-0b27-4036-bbdf-f8374aba68d1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eb0c7bf504fd42c1b1a3e34b3e365060'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:01:28.608368 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-aa08a93d-0b27-4036-bbdf-f8374aba68d1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eb0c7bf504fd42c1b1a3e34b3e365060'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:01:28.615217 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 15:01:28.624416 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1196/2381] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Sep 1 15:01:28 2022] DELETE /identity/v3/users/eb0c7bf504fd42c1b1a3e34b3e365060 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 15:01:28.629466 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6468a962-b05f-4532-8a82-286391d0fe43 None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:01:28.630020 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6468a962-b05f-4532-8a82-286391d0fe43 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:01:28.630020 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6468a962-b05f-4532-8a82-286391d0fe43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:01:28.651596 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-6468a962-b05f-4532-8a82-286391d0fe43 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 15:01:28.652793 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-6468a962-b05f-4532-8a82-286391d0fe43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:01:28.717880 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1186/2382] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 15:01:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 15:01:28.728294 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa08a93d-0b27-4036-bbdf-f8374aba68d1 admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:01:28.729754 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-aa08a93d-0b27-4036-bbdf-f8374aba68d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:01:28.795962 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa08a93d-0b27-4036-bbdf-f8374aba68d1 admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:01:28.799857 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c50a53c-612c-4d27-bbf6-aa0418531dbd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2790149639aa418bbc4e4982e93eda44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:01:28.800758 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c50a53c-612c-4d27-bbf6-aa0418531dbd service neutron] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:01:28.801034 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c50a53c-612c-4d27-bbf6-aa0418531dbd service neutron] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:01:28.801359 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c50a53c-612c-4d27-bbf6-aa0418531dbd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:01:28.810873 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c50a53c-612c-4d27-bbf6-aa0418531dbd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:01:28.812324 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c50a53c-612c-4d27-bbf6-aa0418531dbd service neutron] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:01:28.822298 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1197/2383] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 15:01:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 15:01:29.103723 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6468a962-b05f-4532-8a82-286391d0fe43 None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:01:29.105273 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-6468a962-b05f-4532-8a82-286391d0fe43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:01:29.122225 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6468a962-b05f-4532-8a82-286391d0fe43 None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:01:29.128872 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64342d52-d394-4217-acb7-3c27b88803ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:01:29.129605 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64342d52-d394-4217-acb7-3c27b88803ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:01:29.130107 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64342d52-d394-4217-acb7-3c27b88803ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:01:29.130466 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64342d52-d394-4217-acb7-3c27b88803ba admin admin] PATH_INFO: `/v3/projects/a1a751ba0d92407a80cece63c25b1f24` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:01:29.131832 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64342d52-d394-4217-acb7-3c27b88803ba admin admin] RBAC: Authorizing `identity:delete_project(project_id=a1a751ba0d92407a80cece63c25b1f24)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:01:29.135665 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64342d52-d394-4217-acb7-3c27b88803ba admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:01:29.203025 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-64342d52-d394-4217-acb7-3c27b88803ba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a1a751ba0d92407a80cece63c25b1f24', 'request_id': 'req-64342d52-d394-4217-acb7-3c27b88803ba'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:01:29.212873 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-64342d52-d394-4217-acb7-3c27b88803ba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a1a751ba0d92407a80cece63c25b1f24', 'request_id': 'req-64342d52-d394-4217-acb7-3c27b88803ba'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:01:29.225128 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-64342d52-d394-4217-acb7-3c27b88803ba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a1a751ba0d92407a80cece63c25b1f24', 'request_id': 'req-64342d52-d394-4217-acb7-3c27b88803ba'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:01:29.230825 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 15:01:29.243368 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-64342d52-d394-4217-acb7-3c27b88803ba admin admin] The token cache is being invalidate because project a1a751ba0d92407a80cece63c25b1f24 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81341) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 15:01:29.243863 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-64342d52-d394-4217-acb7-3c27b88803ba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:01:29.244684 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-64342d52-d394-4217-acb7-3c27b88803ba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:01:29.250752 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c50a53c-612c-4d27-bbf6-aa0418531dbd service neutron] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:01:29.251412 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1187/2384] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Sep 1 15:01:29 2022] DELETE /identity/v3/projects/a1a751ba0d92407a80cece63c25b1f24 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 15:01:29.266862 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c50a53c-612c-4d27-bbf6-aa0418531dbd service neutron] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:01:29.367635 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018ad6e5-b6e0-41f0-9b18-ff9408903cfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:01:29.368277 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018ad6e5-b6e0-41f0-9b18-ff9408903cfc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:01:29.368551 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018ad6e5-b6e0-41f0-9b18-ff9408903cfc admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:01:29.368823 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018ad6e5-b6e0-41f0-9b18-ff9408903cfc admin admin] PATH_INFO: `/v3/users/c5c6e682a4ef4f33aeb24f03c712370f` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:01:29.369981 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018ad6e5-b6e0-41f0-9b18-ff9408903cfc admin admin] RBAC: Authorizing `identity:delete_user(user_id=c5c6e682a4ef4f33aeb24f03c712370f)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:01:29.372647 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018ad6e5-b6e0-41f0-9b18-ff9408903cfc admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:01:29.434968 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-018ad6e5-b6e0-41f0-9b18-ff9408903cfc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c5c6e682a4ef4f33aeb24f03c712370f'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:01:29.445877 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-018ad6e5-b6e0-41f0-9b18-ff9408903cfc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c5c6e682a4ef4f33aeb24f03c712370f'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:01:29.450901 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-018ad6e5-b6e0-41f0-9b18-ff9408903cfc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c5c6e682a4ef4f33aeb24f03c712370f'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:01:29.464877 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-018ad6e5-b6e0-41f0-9b18-ff9408903cfc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c5c6e682a4ef4f33aeb24f03c712370f'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:01:29.465663 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-018ad6e5-b6e0-41f0-9b18-ff9408903cfc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c5c6e682a4ef4f33aeb24f03c712370f'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:01:29.481485 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1198/2385] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Sep 1 15:01:29 2022] DELETE /identity/v3/users/c5c6e682a4ef4f33aeb24f03c712370f => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 15:01:29.765195 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64342d52-d394-4217-acb7-3c27b88803ba admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:01:29.767767 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-64342d52-d394-4217-acb7-3c27b88803ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:01:29.792573 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64342d52-d394-4217-acb7-3c27b88803ba admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:01:29.853891 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58693142-0400-4066-9a25-08562a7863f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:01:29.854562 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58693142-0400-4066-9a25-08562a7863f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:01:29.854838 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58693142-0400-4066-9a25-08562a7863f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:01:29.855108 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58693142-0400-4066-9a25-08562a7863f1 admin admin] PATH_INFO: `/v3/projects/c2e8733b7dd847efa524c35ffb316deb` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:01:29.856236 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58693142-0400-4066-9a25-08562a7863f1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c2e8733b7dd847efa524c35ffb316deb)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:01:29.858162 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58693142-0400-4066-9a25-08562a7863f1 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:01:29.903692 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-58693142-0400-4066-9a25-08562a7863f1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c2e8733b7dd847efa524c35ffb316deb', 'request_id': 'req-58693142-0400-4066-9a25-08562a7863f1'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:01:29.910073 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-58693142-0400-4066-9a25-08562a7863f1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c2e8733b7dd847efa524c35ffb316deb', 'request_id': 'req-58693142-0400-4066-9a25-08562a7863f1'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:01:29.918759 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-58693142-0400-4066-9a25-08562a7863f1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c2e8733b7dd847efa524c35ffb316deb', 'request_id': 'req-58693142-0400-4066-9a25-08562a7863f1'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:01:29.930491 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-58693142-0400-4066-9a25-08562a7863f1 admin admin] The token cache is being invalidate because project c2e8733b7dd847efa524c35ffb316deb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81341) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 15:01:29.930785 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-58693142-0400-4066-9a25-08562a7863f1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:01:29.932071 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-58693142-0400-4066-9a25-08562a7863f1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:01:29.946290 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1188/2386] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Sep 1 15:01:29 2022] DELETE /identity/v3/projects/c2e8733b7dd847efa524c35ffb316deb => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 15:01:36.806900 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeaf7c17-252e-4453-a54d-fb347aaca56d None None] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:01:36.807434 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeaf7c17-252e-4453-a54d-fb347aaca56d None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:01:36.807434 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeaf7c17-252e-4453-a54d-fb347aaca56d None None] PATH_INFO: `/` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:01:36.808916 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1199/2387] 10.222.0.33 () {54 vars in 961 bytes} [Thu Sep 1 15:01:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 15:01:36.813868 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83accc2f-6a72-4ec8-a502-86a7c636b428 None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:01:36.814374 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83accc2f-6a72-4ec8-a502-86a7c636b428 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:01:36.814478 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83accc2f-6a72-4ec8-a502-86a7c636b428 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:01:36.834833 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-83accc2f-6a72-4ec8-a502-86a7c636b428 None None] MFA Rules not processed for user `7f312c8dcabe48ef9ea453364ade38ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 15:01:36.836029 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-83accc2f-6a72-4ec8-a502-86a7c636b428 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:01:36.906322 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1189/2388] 10.222.0.33 () {58 vars in 1058 bytes} [Thu Sep 1 15:01:36 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 15:01:36.914171 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeaf7c17-252e-4453-a54d-fb347aaca56d None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:01:36.928272 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeaf7c17-252e-4453-a54d-fb347aaca56d None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:01:36.933236 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cfd52d-6fa4-4a70-a343-884b34341f16 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f312c8dcabe48ef9ea453364ade38ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:01:36.933830 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cfd52d-6fa4-4a70-a343-884b34341f16 service placement] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:01:36.934108 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cfd52d-6fa4-4a70-a343-884b34341f16 service placement] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:01:36.934427 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cfd52d-6fa4-4a70-a343-884b34341f16 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:01:36.936159 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-23cfd52d-6fa4-4a70-a343-884b34341f16 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:01:36.947835 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cfd52d-6fa4-4a70-a343-884b34341f16 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:01:36.949416 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cfd52d-6fa4-4a70-a343-884b34341f16 service placement] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:01:36.959097 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1200/2389] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 15:01:36 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 15:01:46.897733 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:DELETE=8,keystone:SELECT=82,keystone:INSERT=1 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 15:01:46.954952 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 15:01:48.708466 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc900bb-343a-4f89-8f75-b4e233618c3b None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:01:48.708951 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc900bb-343a-4f89-8f75-b4e233618c3b None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:01:48.709102 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc900bb-343a-4f89-8f75-b4e233618c3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:01:48.739570 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-8dc900bb-343a-4f89-8f75-b4e233618c3b None None] MFA Rules not processed for user `24b6f275ffb04ffeae3c2097bfd040f5`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 15:01:48.741369 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-8dc900bb-343a-4f89-8f75-b4e233618c3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:01:48.825878 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1190/2390] 10.222.0.33 () {58 vars in 967 bytes} [Thu Sep 1 15:01:48 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 15:01:48.838859 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cfd52d-6fa4-4a70-a343-884b34341f16 service placement] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:01:48.840329 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-23cfd52d-6fa4-4a70-a343-884b34341f16 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:01:48.909250 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cfd52d-6fa4-4a70-a343-884b34341f16 service placement] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:01:48.914925 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bad19c-93f9-4d84-963f-65913c970d67 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73a5bda5a6ea46e9863ccafd57dee339', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:01:48.915524 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bad19c-93f9-4d84-963f-65913c970d67 service swift] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:01:48.915805 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bad19c-93f9-4d84-963f-65913c970d67 service swift] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:01:48.916076 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bad19c-93f9-4d84-963f-65913c970d67 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:01:48.923791 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1bad19c-93f9-4d84-963f-65913c970d67 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:01:48.925617 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1bad19c-93f9-4d84-963f-65913c970d67 service swift] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:01:48.933366 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1201/2391] 10.222.0.33 () {60 vars in 1384 bytes} [Thu Sep 1 15:01:48 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 01 15:01:58.159383 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64eb188d-b91a-4bd0-ba3b-cbefb97ba4b4 None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:01:58.159791 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64eb188d-b91a-4bd0-ba3b-cbefb97ba4b4 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:01:58.159901 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64eb188d-b91a-4bd0-ba3b-cbefb97ba4b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:01:58.180797 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-64eb188d-b91a-4bd0-ba3b-cbefb97ba4b4 None None] MFA Rules not processed for user `24b6f275ffb04ffeae3c2097bfd040f5`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 15:01:58.182409 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-64eb188d-b91a-4bd0-ba3b-cbefb97ba4b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:01:58.190145 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1191/2392] 10.222.0.33 () {58 vars in 1009 bytes} [Thu Sep 1 15:01:58 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 15:01:58.202779 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bad19c-93f9-4d84-963f-65913c970d67 service swift] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:01:58.213823 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bad19c-93f9-4d84-963f-65913c970d67 service swift] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:01:58.219148 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5beea843-4215-46ab-bb4a-10e76255a623 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73a5bda5a6ea46e9863ccafd57dee339', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:01:58.219746 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5beea843-4215-46ab-bb4a-10e76255a623 service swift] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:01:58.220025 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5beea843-4215-46ab-bb4a-10e76255a623 service swift] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:01:58.220296 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5beea843-4215-46ab-bb4a-10e76255a623 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:01:58.230652 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5beea843-4215-46ab-bb4a-10e76255a623 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:01:58.232276 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5beea843-4215-46ab-bb4a-10e76255a623 service swift] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:01:58.240586 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1202/2393] 10.222.0.33 () {60 vars in 1384 bytes} [Thu Sep 1 15:01:58 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 01 15:02:08.179199 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=28 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 15:02:08.237208 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=38 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 15:02:16.687231 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2337f90-4aba-46da-985e-c5524355d08e None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:02:16.687673 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2337f90-4aba-46da-985e-c5524355d08e None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:02:16.687778 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2337f90-4aba-46da-985e-c5524355d08e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:02:16.707002 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-f2337f90-4aba-46da-985e-c5524355d08e None None] MFA Rules not processed for user `24b6f275ffb04ffeae3c2097bfd040f5`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 15:02:16.708206 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-f2337f90-4aba-46da-985e-c5524355d08e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:02:16.715929 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1192/2394] 10.222.0.33 () {58 vars in 967 bytes} [Thu Sep 1 15:02:16 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 15:02:16.728257 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5beea843-4215-46ab-bb4a-10e76255a623 service swift] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:02:16.738522 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5beea843-4215-46ab-bb4a-10e76255a623 service swift] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:02:16.743444 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb228fa-c364-474f-aae8-ab0a4b959227 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73a5bda5a6ea46e9863ccafd57dee339', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:02:16.744028 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb228fa-c364-474f-aae8-ab0a4b959227 service swift] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:02:16.744306 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb228fa-c364-474f-aae8-ab0a4b959227 service swift] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:02:16.744579 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb228fa-c364-474f-aae8-ab0a4b959227 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:02:16.752355 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb228fa-c364-474f-aae8-ab0a4b959227 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:02:16.753915 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb228fa-c364-474f-aae8-ab0a4b959227 service swift] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:02:16.762268 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1203/2395] 10.222.0.33 () {60 vars in 1384 bytes} [Thu Sep 1 15:02:16 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 01 15:02:16.965759 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbc4f38-a7ba-4ad9-9d0b-d152f0e15713 None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:02:16.966182 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbc4f38-a7ba-4ad9-9d0b-d152f0e15713 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:02:16.966317 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbc4f38-a7ba-4ad9-9d0b-d152f0e15713 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:02:16.986230 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-bbbc4f38-a7ba-4ad9-9d0b-d152f0e15713 None None] MFA Rules not processed for user `74b4bf3f69204fc896c996bcaa9e087e`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 15:02:16.987470 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-bbbc4f38-a7ba-4ad9-9d0b-d152f0e15713 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:02:17.044726 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1193/2396] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 15:02:16 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 15:02:18.822640 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb228fa-c364-474f-aae8-ab0a4b959227 service swift] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:02:18.824119 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-2eb228fa-c364-474f-aae8-ab0a4b959227 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:02:18.904755 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb228fa-c364-474f-aae8-ab0a4b959227 service swift] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:02:18.909747 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43e7bce-8ba3-4764-ae07-7daa690c3681 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:02:18.910560 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43e7bce-8ba3-4764-ae07-7daa690c3681 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:02:18.910944 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43e7bce-8ba3-4764-ae07-7daa690c3681 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:02:18.911447 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43e7bce-8ba3-4764-ae07-7daa690c3681 admin admin] PATH_INFO: `/v3/users/534901c491b64bbe8bdc59776e643a78` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:02:18.912831 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43e7bce-8ba3-4764-ae07-7daa690c3681 admin admin] RBAC: Authorizing `identity:delete_user(user_id=534901c491b64bbe8bdc59776e643a78)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:02:18.915440 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43e7bce-8ba3-4764-ae07-7daa690c3681 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:02:18.970641 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-f43e7bce-8ba3-4764-ae07-7daa690c3681 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '534901c491b64bbe8bdc59776e643a78'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:02:18.980424 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-f43e7bce-8ba3-4764-ae07-7daa690c3681 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '534901c491b64bbe8bdc59776e643a78'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:02:18.981782 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-f43e7bce-8ba3-4764-ae07-7daa690c3681 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '534901c491b64bbe8bdc59776e643a78'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:02:18.996950 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-f43e7bce-8ba3-4764-ae07-7daa690c3681 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '534901c491b64bbe8bdc59776e643a78'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:02:18.997718 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-f43e7bce-8ba3-4764-ae07-7daa690c3681 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '534901c491b64bbe8bdc59776e643a78'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:02:19.011395 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1204/2397] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Sep 1 15:02:18 2022] DELETE /identity/v3/users/534901c491b64bbe8bdc59776e643a78 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 15:02:19.193158 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbc4f38-a7ba-4ad9-9d0b-d152f0e15713 None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:02:19.206782 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbc4f38-a7ba-4ad9-9d0b-d152f0e15713 None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:02:19.277943 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb012eb-ff5d-423d-b438-1df348841cc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:02:19.278909 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb012eb-ff5d-423d-b438-1df348841cc4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:02:19.279185 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb012eb-ff5d-423d-b438-1df348841cc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:02:19.279453 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb012eb-ff5d-423d-b438-1df348841cc4 admin admin] PATH_INFO: `/v3/projects/ba5a5ea6aa4b4736ae49915583626e18` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:02:19.280598 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb012eb-ff5d-423d-b438-1df348841cc4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ba5a5ea6aa4b4736ae49915583626e18)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:02:19.282548 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb012eb-ff5d-423d-b438-1df348841cc4 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:02:19.329908 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-4cb012eb-ff5d-423d-b438-1df348841cc4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ba5a5ea6aa4b4736ae49915583626e18', 'request_id': 'req-4cb012eb-ff5d-423d-b438-1df348841cc4'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:02:19.335993 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-4cb012eb-ff5d-423d-b438-1df348841cc4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ba5a5ea6aa4b4736ae49915583626e18', 'request_id': 'req-4cb012eb-ff5d-423d-b438-1df348841cc4'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:02:19.345721 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-4cb012eb-ff5d-423d-b438-1df348841cc4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ba5a5ea6aa4b4736ae49915583626e18', 'request_id': 'req-4cb012eb-ff5d-423d-b438-1df348841cc4'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:02:19.357205 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-4cb012eb-ff5d-423d-b438-1df348841cc4 admin admin] The token cache is being invalidate because project ba5a5ea6aa4b4736ae49915583626e18 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81341) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 15:02:19.357496 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-4cb012eb-ff5d-423d-b438-1df348841cc4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:02:19.358136 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-4cb012eb-ff5d-423d-b438-1df348841cc4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:02:19.360663 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1194/2398] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Sep 1 15:02:19 2022] DELETE /identity/v3/projects/ba5a5ea6aa4b4736ae49915583626e18 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 15:02:19.366580 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23662bbd-1242-410e-b5bf-2dbf7d9d9b29 None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:02:19.366869 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23662bbd-1242-410e-b5bf-2dbf7d9d9b29 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:02:19.367148 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23662bbd-1242-410e-b5bf-2dbf7d9d9b29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:02:19.386907 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-23662bbd-1242-410e-b5bf-2dbf7d9d9b29 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 15:02:19.388247 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-23662bbd-1242-410e-b5bf-2dbf7d9d9b29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:02:19.457411 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1205/2399] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 15:02:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 15:02:19.459416 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e83085-5ace-41fa-9e64-ba8483ff1e99 None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:02:19.459776 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e83085-5ace-41fa-9e64-ba8483ff1e99 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:02:19.460191 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e83085-5ace-41fa-9e64-ba8483ff1e99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:02:19.481361 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-05e83085-5ace-41fa-9e64-ba8483ff1e99 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 15:02:19.482712 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-05e83085-5ace-41fa-9e64-ba8483ff1e99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:02:19.490980 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1195/2400] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 15:02:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 15:02:19.495509 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23662bbd-1242-410e-b5bf-2dbf7d9d9b29 None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:02:19.505068 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23662bbd-1242-410e-b5bf-2dbf7d9d9b29 None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:02:19.509699 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a25f60-6ddd-4ac2-bdd6-bbdcf8821c4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:02:19.510796 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a25f60-6ddd-4ac2-bdd6-bbdcf8821c4d admin admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:02:19.510796 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a25f60-6ddd-4ac2-bdd6-bbdcf8821c4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:02:19.510997 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a25f60-6ddd-4ac2-bdd6-bbdcf8821c4d admin admin] PATH_INFO: `/v3/domains` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:02:19.511483 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a25f60-6ddd-4ac2-bdd6-bbdcf8821c4d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 15:02:19.511780 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a25f60-6ddd-4ac2-bdd6-bbdcf8821c4d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:02:19.513407 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a25f60-6ddd-4ac2-bdd6-bbdcf8821c4d admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:02:19.514929 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 15:02:19.527715 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1206/2401] 10.222.0.33 () {58 vars in 1153 bytes} [Thu Sep 1 15:02:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 15:02:19.531809 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e83085-5ace-41fa-9e64-ba8483ff1e99 None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:02:19.541552 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e83085-5ace-41fa-9e64-ba8483ff1e99 None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:02:19.546326 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6206f505-f874-4fb1-9743-68688c074d76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:02:19.546953 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6206f505-f874-4fb1-9743-68688c074d76 admin admin] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:02:19.547228 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6206f505-f874-4fb1-9743-68688c074d76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:02:19.547493 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6206f505-f874-4fb1-9743-68688c074d76 admin admin] PATH_INFO: `/v3/projects` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:02:19.548116 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6206f505-f874-4fb1-9743-68688c074d76 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:02:19.550354 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6206f505-f874-4fb1-9743-68688c074d76 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:02:19.571821 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 15:02:19.579895 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1196/2402] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Sep 1 15:02:19 2022] POST /identity/v3/projects => generated 375 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 15:02:19.583601 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a25f60-6ddd-4ac2-bdd6-bbdcf8821c4d admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:02:19.594875 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a25f60-6ddd-4ac2-bdd6-bbdcf8821c4d admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:02:19.675772 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bf802d-57f4-4fae-a4fa-5536be7b4303 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:02:19.676409 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bf802d-57f4-4fae-a4fa-5536be7b4303 admin admin] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:02:19.676692 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bf802d-57f4-4fae-a4fa-5536be7b4303 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:02:19.676967 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bf802d-57f4-4fae-a4fa-5536be7b4303 admin admin] PATH_INFO: `/v3/users` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:02:19.677609 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1bf802d-57f4-4fae-a4fa-5536be7b4303 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:02:19.680211 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1bf802d-57f4-4fae-a4fa-5536be7b4303 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:02:19.721700 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1207/2403] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Sep 1 15:02:19 2022] POST /identity/v3/users => generated 288 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 15:02:19.725887 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6206f505-f874-4fb1-9743-68688c074d76 admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:02:19.736935 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6206f505-f874-4fb1-9743-68688c074d76 admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:02:19.742694 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf73af0-2224-4fe1-9c0b-2c5cd92294a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:02:19.743303 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf73af0-2224-4fe1-9c0b-2c5cd92294a1 admin admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:02:19.743584 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf73af0-2224-4fe1-9c0b-2c5cd92294a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:02:19.743854 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf73af0-2224-4fe1-9c0b-2c5cd92294a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:02:19.744554 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf73af0-2224-4fe1-9c0b-2c5cd92294a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:02:19.747012 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf73af0-2224-4fe1-9c0b-2c5cd92294a1 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:02:19.760116 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1197/2404] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 1 15:02:19 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 15:02:19.764145 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bf802d-57f4-4fae-a4fa-5536be7b4303 admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:02:19.775389 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bf802d-57f4-4fae-a4fa-5536be7b4303 admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:02:19.782299 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8eb55a3-3c2d-4fb3-9101-b1c9f46022b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:02:19.783227 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8eb55a3-3c2d-4fb3-9101-b1c9f46022b4 admin admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:02:19.783520 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8eb55a3-3c2d-4fb3-9101-b1c9f46022b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:02:19.783791 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8eb55a3-3c2d-4fb3-9101-b1c9f46022b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:02:19.784955 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8eb55a3-3c2d-4fb3-9101-b1c9f46022b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:02:19.787059 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8eb55a3-3c2d-4fb3-9101-b1c9f46022b4 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:02:19.800483 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1208/2405] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 1 15:02:19 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 15:02:19.804477 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf73af0-2224-4fe1-9c0b-2c5cd92294a1 admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:02:19.816750 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf73af0-2224-4fe1-9c0b-2c5cd92294a1 admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:02:19.824251 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438fa6d5-1997-4195-8a64-03156f866a3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:02:19.825074 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438fa6d5-1997-4195-8a64-03156f866a3e admin admin] REQUEST_METHOD: `PUT` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:02:19.825379 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438fa6d5-1997-4195-8a64-03156f866a3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:02:19.825670 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438fa6d5-1997-4195-8a64-03156f866a3e admin admin] PATH_INFO: `/v3/projects/a4a3abc1fc9e42c5a1d634187cb0e21e/users/2925efdc72544fbe9131ef44359958ad/roles/5d1feef0e13d4874928bd86258c4e3b2` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:02:19.850076 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438fa6d5-1997-4195-8a64-03156f866a3e admin admin] RBAC: Authorizing `identity:create_grant(project_id=a4a3abc1fc9e42c5a1d634187cb0e21e, user_id=2925efdc72544fbe9131ef44359958ad, role_id=5d1feef0e13d4874928bd86258c4e3b2)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:02:19.852896 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438fa6d5-1997-4195-8a64-03156f866a3e admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:02:19.868729 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1198/2406] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 1 15:02:19 2022] PUT /identity/v3/projects/a4a3abc1fc9e42c5a1d634187cb0e21e/users/2925efdc72544fbe9131ef44359958ad/roles/5d1feef0e13d4874928bd86258c4e3b2 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 15:02:19.880317 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6cf912-8ffc-471e-826a-99ad1eecf275 None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:02:19.880891 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6cf912-8ffc-471e-826a-99ad1eecf275 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:02:19.880891 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6cf912-8ffc-471e-826a-99ad1eecf275 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:02:19.911773 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-8e6cf912-8ffc-471e-826a-99ad1eecf275 None None] MFA Rules not processed for user `2925efdc72544fbe9131ef44359958ad`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 15:02:19.913010 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-8e6cf912-8ffc-471e-826a-99ad1eecf275 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:02:20.003480 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1209/2407] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 15:02:19 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 15:02:20.016024 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438fa6d5-1997-4195-8a64-03156f866a3e admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:02:20.017917 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-438fa6d5-1997-4195-8a64-03156f866a3e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:02:20.102004 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438fa6d5-1997-4195-8a64-03156f866a3e admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:02:20.108922 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecdc269-0bcc-49a5-93c4-e8955f797cfd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3b81739e34e4e648f88b6d73fcb5e46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:02:20.109711 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecdc269-0bcc-49a5-93c4-e8955f797cfd service cinder] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:02:20.110067 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecdc269-0bcc-49a5-93c4-e8955f797cfd service cinder] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:02:20.110436 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecdc269-0bcc-49a5-93c4-e8955f797cfd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:02:20.123044 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecdc269-0bcc-49a5-93c4-e8955f797cfd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:02:20.125011 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecdc269-0bcc-49a5-93c4-e8955f797cfd service cinder] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:02:20.135277 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1199/2408] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 15:02:20 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 15:02:29.999307 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=69,keystone:INSERT=3 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 15:02:30.130328 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=41,keystone:INSERT=1 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 15:02:30.278981 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6cf912-8ffc-471e-826a-99ad1eecf275 None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:02:30.362517 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6cf912-8ffc-471e-826a-99ad1eecf275 None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:02:30.368801 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0acaeef-c205-459e-87e5-8862d1b0c3d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:02:30.369622 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0acaeef-c205-459e-87e5-8862d1b0c3d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:02:30.370036 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0acaeef-c205-459e-87e5-8862d1b0c3d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:02:30.370855 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0acaeef-c205-459e-87e5-8862d1b0c3d9 admin admin] PATH_INFO: `/v3/users/2925efdc72544fbe9131ef44359958ad` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:02:30.372002 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0acaeef-c205-459e-87e5-8862d1b0c3d9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2925efdc72544fbe9131ef44359958ad)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:02:30.374456 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0acaeef-c205-459e-87e5-8862d1b0c3d9 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:02:30.451463 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-e0acaeef-c205-459e-87e5-8862d1b0c3d9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2925efdc72544fbe9131ef44359958ad'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:02:30.459314 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-e0acaeef-c205-459e-87e5-8862d1b0c3d9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2925efdc72544fbe9131ef44359958ad'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:02:30.459980 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-e0acaeef-c205-459e-87e5-8862d1b0c3d9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2925efdc72544fbe9131ef44359958ad'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:02:30.472746 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-e0acaeef-c205-459e-87e5-8862d1b0c3d9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2925efdc72544fbe9131ef44359958ad'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:02:30.473456 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-e0acaeef-c205-459e-87e5-8862d1b0c3d9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2925efdc72544fbe9131ef44359958ad'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:02:30.486625 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1210/2409] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Sep 1 15:02:30 2022] DELETE /identity/v3/users/2925efdc72544fbe9131ef44359958ad => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 15:02:30.491258 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4246aa7-4851-4063-a71b-e7a42da85b32 None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:02:30.491547 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4246aa7-4851-4063-a71b-e7a42da85b32 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:02:30.491819 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4246aa7-4851-4063-a71b-e7a42da85b32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:02:30.511430 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-c4246aa7-4851-4063-a71b-e7a42da85b32 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 15:02:30.512643 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-c4246aa7-4851-4063-a71b-e7a42da85b32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:02:30.571169 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1200/2410] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 15:02:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 15:02:30.580939 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0acaeef-c205-459e-87e5-8862d1b0c3d9 admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:02:30.582384 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-e0acaeef-c205-459e-87e5-8862d1b0c3d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:02:30.639656 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0acaeef-c205-459e-87e5-8862d1b0c3d9 admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:02:30.643274 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3a2e02-86f5-411e-9935-2c96ed9019f1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2790149639aa418bbc4e4982e93eda44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:02:30.643871 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3a2e02-86f5-411e-9935-2c96ed9019f1 service neutron] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:02:30.644150 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3a2e02-86f5-411e-9935-2c96ed9019f1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:02:30.644423 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3a2e02-86f5-411e-9935-2c96ed9019f1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:02:30.654724 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3a2e02-86f5-411e-9935-2c96ed9019f1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:02:30.656162 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3a2e02-86f5-411e-9935-2c96ed9019f1 service neutron] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:02:30.667599 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1211/2411] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 15:02:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 15:02:31.029889 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4246aa7-4851-4063-a71b-e7a42da85b32 None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:02:31.031479 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-c4246aa7-4851-4063-a71b-e7a42da85b32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:02:31.043993 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4246aa7-4851-4063-a71b-e7a42da85b32 None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:02:31.048742 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26641527-435c-4958-91c1-a2d41a53073f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:02:31.049743 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26641527-435c-4958-91c1-a2d41a53073f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:02:31.050102 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26641527-435c-4958-91c1-a2d41a53073f admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:02:31.050493 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26641527-435c-4958-91c1-a2d41a53073f admin admin] PATH_INFO: `/v3/projects/a4a3abc1fc9e42c5a1d634187cb0e21e` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:02:31.051689 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26641527-435c-4958-91c1-a2d41a53073f admin admin] RBAC: Authorizing `identity:delete_project(project_id=a4a3abc1fc9e42c5a1d634187cb0e21e)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:02:31.053933 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26641527-435c-4958-91c1-a2d41a53073f admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:02:31.100820 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-26641527-435c-4958-91c1-a2d41a53073f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a4a3abc1fc9e42c5a1d634187cb0e21e', 'request_id': 'req-26641527-435c-4958-91c1-a2d41a53073f'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:02:31.106832 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-26641527-435c-4958-91c1-a2d41a53073f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a4a3abc1fc9e42c5a1d634187cb0e21e', 'request_id': 'req-26641527-435c-4958-91c1-a2d41a53073f'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:02:31.113987 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-26641527-435c-4958-91c1-a2d41a53073f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a4a3abc1fc9e42c5a1d634187cb0e21e', 'request_id': 'req-26641527-435c-4958-91c1-a2d41a53073f'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:02:31.124104 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-26641527-435c-4958-91c1-a2d41a53073f admin admin] The token cache is being invalidate because project a4a3abc1fc9e42c5a1d634187cb0e21e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81341) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 15:02:31.124390 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-26641527-435c-4958-91c1-a2d41a53073f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:02:31.125010 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-26641527-435c-4958-91c1-a2d41a53073f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:02:31.126505 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1201/2412] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Sep 1 15:02:31 2022] DELETE /identity/v3/projects/a4a3abc1fc9e42c5a1d634187cb0e21e => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 15:02:31.131290 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578bd14f-04d8-4ae1-ab82-b81500d21fd6 None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:02:31.131803 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578bd14f-04d8-4ae1-ab82-b81500d21fd6 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:02:31.131803 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578bd14f-04d8-4ae1-ab82-b81500d21fd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:02:31.153243 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-578bd14f-04d8-4ae1-ab82-b81500d21fd6 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 15:02:31.154408 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-578bd14f-04d8-4ae1-ab82-b81500d21fd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:02:31.170511 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 15:02:31.213940 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1212/2413] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 15:02:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 15:02:31.221795 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9e43cd-16cb-4c11-b84f-3ce33327ed84 None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:02:31.222281 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9e43cd-16cb-4c11-b84f-3ce33327ed84 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:02:31.222391 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9e43cd-16cb-4c11-b84f-3ce33327ed84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:02:31.241789 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-3a9e43cd-16cb-4c11-b84f-3ce33327ed84 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 15:02:31.243134 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-3a9e43cd-16cb-4c11-b84f-3ce33327ed84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:02:31.251053 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1202/2414] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 15:02:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 15:02:31.255095 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578bd14f-04d8-4ae1-ab82-b81500d21fd6 None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:02:31.264896 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578bd14f-04d8-4ae1-ab82-b81500d21fd6 None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:02:31.269394 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e61807-da82-4d2b-8f6f-3e1a59162679 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:02:31.269979 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e61807-da82-4d2b-8f6f-3e1a59162679 admin admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:02:31.270280 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e61807-da82-4d2b-8f6f-3e1a59162679 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:02:31.270548 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e61807-da82-4d2b-8f6f-3e1a59162679 admin admin] PATH_INFO: `/v3/domains` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:02:31.271182 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e61807-da82-4d2b-8f6f-3e1a59162679 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 15:02:31.271478 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e61807-da82-4d2b-8f6f-3e1a59162679 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:02:31.273061 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e61807-da82-4d2b-8f6f-3e1a59162679 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:02:31.285021 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1213/2415] 10.222.0.33 () {58 vars in 1153 bytes} [Thu Sep 1 15:02:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 15:02:31.288956 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9e43cd-16cb-4c11-b84f-3ce33327ed84 None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:02:31.298032 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9e43cd-16cb-4c11-b84f-3ce33327ed84 None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:02:31.302909 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa8479f-e6a7-4b82-b9cb-ccd4496c6322 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:02:31.303487 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa8479f-e6a7-4b82-b9cb-ccd4496c6322 admin admin] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:02:31.303763 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa8479f-e6a7-4b82-b9cb-ccd4496c6322 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:02:31.304035 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa8479f-e6a7-4b82-b9cb-ccd4496c6322 admin admin] PATH_INFO: `/v3/projects` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:02:31.305170 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa8479f-e6a7-4b82-b9cb-ccd4496c6322 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:02:31.307129 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa8479f-e6a7-4b82-b9cb-ccd4496c6322 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:02:31.333917 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1203/2416] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Sep 1 15:02:31 2022] POST /identity/v3/projects => generated 379 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 15:02:31.337914 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e61807-da82-4d2b-8f6f-3e1a59162679 admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:02:31.347752 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e61807-da82-4d2b-8f6f-3e1a59162679 admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:02:31.402300 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a252e2c0-83d4-4a2c-a8e1-1cf4fdb73b9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:02:31.402891 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a252e2c0-83d4-4a2c-a8e1-1cf4fdb73b9b admin admin] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:02:31.403168 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a252e2c0-83d4-4a2c-a8e1-1cf4fdb73b9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:02:31.403483 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a252e2c0-83d4-4a2c-a8e1-1cf4fdb73b9b admin admin] PATH_INFO: `/v3/users` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:02:31.404056 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a252e2c0-83d4-4a2c-a8e1-1cf4fdb73b9b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:02:31.406306 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a252e2c0-83d4-4a2c-a8e1-1cf4fdb73b9b admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:02:31.429070 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1214/2417] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Sep 1 15:02:31 2022] POST /identity/v3/users => generated 290 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 15:02:31.433539 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa8479f-e6a7-4b82-b9cb-ccd4496c6322 admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:02:31.443972 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa8479f-e6a7-4b82-b9cb-ccd4496c6322 admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:02:31.451314 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3846ab4-bf31-4b9e-8df8-c263b235d39a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:02:31.452008 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3846ab4-bf31-4b9e-8df8-c263b235d39a admin admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:02:31.452284 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3846ab4-bf31-4b9e-8df8-c263b235d39a admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:02:31.452553 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3846ab4-bf31-4b9e-8df8-c263b235d39a admin admin] PATH_INFO: `/v3/roles` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:02:31.453197 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3846ab4-bf31-4b9e-8df8-c263b235d39a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:02:31.454841 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3846ab4-bf31-4b9e-8df8-c263b235d39a admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:02:31.465823 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1204/2418] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 1 15:02:31 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 15:02:31.470444 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a252e2c0-83d4-4a2c-a8e1-1cf4fdb73b9b admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:02:31.486867 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a252e2c0-83d4-4a2c-a8e1-1cf4fdb73b9b admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:02:31.491556 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006d191f-8b2d-4e14-9f2f-5f1371016d6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:02:31.492632 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006d191f-8b2d-4e14-9f2f-5f1371016d6a admin admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:02:31.492908 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006d191f-8b2d-4e14-9f2f-5f1371016d6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:02:31.493174 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006d191f-8b2d-4e14-9f2f-5f1371016d6a admin admin] PATH_INFO: `/v3/roles` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:02:31.493812 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006d191f-8b2d-4e14-9f2f-5f1371016d6a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:02:31.495442 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006d191f-8b2d-4e14-9f2f-5f1371016d6a admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:02:31.506906 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1215/2419] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 1 15:02:31 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 15:02:31.511648 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3846ab4-bf31-4b9e-8df8-c263b235d39a admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:02:31.522180 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3846ab4-bf31-4b9e-8df8-c263b235d39a admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:02:31.526684 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03010c69-f2ce-4338-8e20-10cf0c639fa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:02:31.527332 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03010c69-f2ce-4338-8e20-10cf0c639fa5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:02:31.527612 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03010c69-f2ce-4338-8e20-10cf0c639fa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:02:31.527889 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03010c69-f2ce-4338-8e20-10cf0c639fa5 admin admin] PATH_INFO: `/v3/projects/3e49151735934f21a2ea909842e36a05/users/afa6212f31034e27a8d32f114937190e/roles/5d1feef0e13d4874928bd86258c4e3b2` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:02:31.542807 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03010c69-f2ce-4338-8e20-10cf0c639fa5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e49151735934f21a2ea909842e36a05, user_id=afa6212f31034e27a8d32f114937190e, role_id=5d1feef0e13d4874928bd86258c4e3b2)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:02:31.545476 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03010c69-f2ce-4338-8e20-10cf0c639fa5 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:02:31.559676 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1205/2420] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 1 15:02:31 2022] PUT /identity/v3/projects/3e49151735934f21a2ea909842e36a05/users/afa6212f31034e27a8d32f114937190e/roles/5d1feef0e13d4874928bd86258c4e3b2 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 15:02:31.564704 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86012a7e-7cee-46d5-985f-f28b11baaa14 None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:02:31.564995 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86012a7e-7cee-46d5-985f-f28b11baaa14 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:02:31.565285 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86012a7e-7cee-46d5-985f-f28b11baaa14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:02:31.584819 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-86012a7e-7cee-46d5-985f-f28b11baaa14 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 15:02:31.585943 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-86012a7e-7cee-46d5-985f-f28b11baaa14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:02:31.644151 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1216/2421] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 15:02:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 15:02:31.653881 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03010c69-f2ce-4338-8e20-10cf0c639fa5 admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:02:31.655646 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-03010c69-f2ce-4338-8e20-10cf0c639fa5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:02:31.677858 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 15:02:31.714865 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03010c69-f2ce-4338-8e20-10cf0c639fa5 admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:02:31.718517 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67983ce2-859d-4dfd-9be2-b3290a1a461e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2790149639aa418bbc4e4982e93eda44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:02:31.719114 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67983ce2-859d-4dfd-9be2-b3290a1a461e service neutron] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:02:31.719391 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67983ce2-859d-4dfd-9be2-b3290a1a461e service neutron] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:02:31.719658 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67983ce2-859d-4dfd-9be2-b3290a1a461e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:02:31.734833 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67983ce2-859d-4dfd-9be2-b3290a1a461e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:02:31.736297 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67983ce2-859d-4dfd-9be2-b3290a1a461e service neutron] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:02:31.746356 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1206/2422] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 15:02:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 15:02:35.994134 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8032492b-5a79-4768-b0f9-e9ba7d940aac None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:02:35.994707 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8032492b-5a79-4768-b0f9-e9ba7d940aac None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:02:35.994815 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8032492b-5a79-4768-b0f9-e9ba7d940aac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:02:36.038151 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-8032492b-5a79-4768-b0f9-e9ba7d940aac None None] MFA Rules not processed for user `afa6212f31034e27a8d32f114937190e`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 15:02:36.040063 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-8032492b-5a79-4768-b0f9-e9ba7d940aac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:02:36.061090 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 15:02:36.117062 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1217/2423] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 15:02:35 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 15:02:36.121490 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67983ce2-859d-4dfd-9be2-b3290a1a461e service neutron] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:02:36.131519 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67983ce2-859d-4dfd-9be2-b3290a1a461e service neutron] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:02:36.136612 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e218516-e5de-4ee6-8eaa-4a5700e05a13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:02:36.137207 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e218516-e5de-4ee6-8eaa-4a5700e05a13 admin admin] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:02:36.137483 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e218516-e5de-4ee6-8eaa-4a5700e05a13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:02:36.137748 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e218516-e5de-4ee6-8eaa-4a5700e05a13 admin admin] PATH_INFO: `/v3/projects` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:02:36.138419 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e218516-e5de-4ee6-8eaa-4a5700e05a13 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:02:36.140426 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e218516-e5de-4ee6-8eaa-4a5700e05a13 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:02:36.169243 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1207/2424] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Sep 1 15:02:36 2022] POST /identity/v3/projects => generated 379 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 15:02:36.173260 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8032492b-5a79-4768-b0f9-e9ba7d940aac None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:02:36.185064 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8032492b-5a79-4768-b0f9-e9ba7d940aac None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:02:36.255875 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21289a8-2f61-451b-8512-c2e2500530c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:02:36.256564 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21289a8-2f61-451b-8512-c2e2500530c4 admin admin] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:02:36.256910 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21289a8-2f61-451b-8512-c2e2500530c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:02:36.257251 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21289a8-2f61-451b-8512-c2e2500530c4 admin admin] PATH_INFO: `/v3/users` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:02:36.270152 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21289a8-2f61-451b-8512-c2e2500530c4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:02:36.272401 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21289a8-2f61-451b-8512-c2e2500530c4 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:02:36.308105 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1218/2425] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Sep 1 15:02:36 2022] POST /identity/v3/users => generated 296 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 15:02:36.312358 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e218516-e5de-4ee6-8eaa-4a5700e05a13 admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:02:36.323312 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e218516-e5de-4ee6-8eaa-4a5700e05a13 admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:02:36.328188 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d4fcc4-c6eb-4f7e-beba-7b09f7b2b353 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:02:36.328616 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d4fcc4-c6eb-4f7e-beba-7b09f7b2b353 admin admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:02:36.328895 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d4fcc4-c6eb-4f7e-beba-7b09f7b2b353 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:02:36.329166 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d4fcc4-c6eb-4f7e-beba-7b09f7b2b353 admin admin] PATH_INFO: `/v3/roles` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:02:36.329925 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d4fcc4-c6eb-4f7e-beba-7b09f7b2b353 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:02:36.331563 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d4fcc4-c6eb-4f7e-beba-7b09f7b2b353 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:02:36.347311 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1208/2426] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 1 15:02:36 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 15:02:36.348247 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21289a8-2f61-451b-8512-c2e2500530c4 admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:02:36.359376 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21289a8-2f61-451b-8512-c2e2500530c4 admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:02:36.364297 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69f0453-a7b9-4f1d-852a-3b8c3d128e21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:02:36.365005 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69f0453-a7b9-4f1d-852a-3b8c3d128e21 admin admin] REQUEST_METHOD: `PUT` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:02:36.365282 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69f0453-a7b9-4f1d-852a-3b8c3d128e21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:02:36.365550 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69f0453-a7b9-4f1d-852a-3b8c3d128e21 admin admin] PATH_INFO: `/v3/projects/32962f5671bb4bbfa75d1d8efb7f684a/users/774adf3c3da84a78979a826f6f884922/roles/2a2adb769e55476b82e5014897e0a9c0` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:02:36.384521 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69f0453-a7b9-4f1d-852a-3b8c3d128e21 admin admin] RBAC: Authorizing `identity:create_grant(project_id=32962f5671bb4bbfa75d1d8efb7f684a, user_id=774adf3c3da84a78979a826f6f884922, role_id=2a2adb769e55476b82e5014897e0a9c0)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:02:36.387843 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69f0453-a7b9-4f1d-852a-3b8c3d128e21 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:02:36.404899 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1219/2427] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 1 15:02:36 2022] PUT /identity/v3/projects/32962f5671bb4bbfa75d1d8efb7f684a/users/774adf3c3da84a78979a826f6f884922/roles/2a2adb769e55476b82e5014897e0a9c0 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 15:02:36.408886 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d4fcc4-c6eb-4f7e-beba-7b09f7b2b353 admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:02:36.419680 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d4fcc4-c6eb-4f7e-beba-7b09f7b2b353 admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:02:36.485938 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4aabd4a-e74e-41ca-9c58-e5a8833e12b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:02:36.486579 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4aabd4a-e74e-41ca-9c58-e5a8833e12b0 admin admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:02:36.486854 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4aabd4a-e74e-41ca-9c58-e5a8833e12b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:02:36.487122 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4aabd4a-e74e-41ca-9c58-e5a8833e12b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:02:36.487820 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4aabd4a-e74e-41ca-9c58-e5a8833e12b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:02:36.489566 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4aabd4a-e74e-41ca-9c58-e5a8833e12b0 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:02:36.510416 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69f0453-a7b9-4f1d-852a-3b8c3d128e21 admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:02:36.510641 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1209/2428] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 1 15:02:36 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 15:02:36.528805 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69f0453-a7b9-4f1d-852a-3b8c3d128e21 admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:02:36.534200 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f68df52-ac41-49d7-a4af-db5997f867ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:02:36.535009 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f68df52-ac41-49d7-a4af-db5997f867ef admin admin] REQUEST_METHOD: `PUT` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:02:36.535750 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f68df52-ac41-49d7-a4af-db5997f867ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:02:36.535750 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f68df52-ac41-49d7-a4af-db5997f867ef admin admin] PATH_INFO: `/v3/projects/32962f5671bb4bbfa75d1d8efb7f684a/users/774adf3c3da84a78979a826f6f884922/roles/2a2adb769e55476b82e5014897e0a9c0` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:02:36.537687 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f68df52-ac41-49d7-a4af-db5997f867ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=32962f5671bb4bbfa75d1d8efb7f684a, user_id=774adf3c3da84a78979a826f6f884922, role_id=2a2adb769e55476b82e5014897e0a9c0)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:02:36.540726 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f68df52-ac41-49d7-a4af-db5997f867ef admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:02:36.558386 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1220/2429] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 1 15:02:36 2022] PUT /identity/v3/projects/32962f5671bb4bbfa75d1d8efb7f684a/users/774adf3c3da84a78979a826f6f884922/roles/2a2adb769e55476b82e5014897e0a9c0 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 15:02:42.041246 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43522ef3-ec60-4835-8d3b-fb5a56cbdb29 None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:02:42.041859 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43522ef3-ec60-4835-8d3b-fb5a56cbdb29 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:02:42.042477 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43522ef3-ec60-4835-8d3b-fb5a56cbdb29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:02:42.101029 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-43522ef3-ec60-4835-8d3b-fb5a56cbdb29 None None] MFA Rules not processed for user `774adf3c3da84a78979a826f6f884922`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 15:02:42.103306 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-43522ef3-ec60-4835-8d3b-fb5a56cbdb29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:02:42.184383 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1210/2430] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 15:02:42 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 15:02:42.198000 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f68df52-ac41-49d7-a4af-db5997f867ef admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:02:42.200132 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-4f68df52-ac41-49d7-a4af-db5997f867ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:02:42.318552 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f68df52-ac41-49d7-a4af-db5997f867ef admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:02:42.325925 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acdb2e8-c43f-4860-909a-5500104fd2c3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4858bb8621854ec8aba836e7d9050498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:02:42.326579 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acdb2e8-c43f-4860-909a-5500104fd2c3 service nova] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:02:42.326862 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acdb2e8-c43f-4860-909a-5500104fd2c3 service nova] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:02:42.327134 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acdb2e8-c43f-4860-909a-5500104fd2c3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:02:42.332665 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 15:02:42.405745 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acdb2e8-c43f-4860-909a-5500104fd2c3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:02:42.407460 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acdb2e8-c43f-4860-909a-5500104fd2c3 service nova] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:02:42.419360 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1221/2431] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 15:02:42 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 15:02:49.062389 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8d52fc-211d-4685-b178-0a83ff60c489 None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:02:49.062389 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8d52fc-211d-4685-b178-0a83ff60c489 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:02:49.062389 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8d52fc-211d-4685-b178-0a83ff60c489 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:02:49.075867 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-dd8d52fc-211d-4685-b178-0a83ff60c489 None None] MFA Rules not processed for user `df75151c37f74b6da7428dc290a274ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 15:02:49.077061 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-dd8d52fc-211d-4685-b178-0a83ff60c489 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:02:49.082748 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 15:02:49.146334 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1211/2432] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 15:02:49 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 15:02:52.415776 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=24 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 15:02:52.446449 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c27b7c-76e4-4d87-9885-e32851152327 None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:02:52.446805 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c27b7c-76e4-4d87-9885-e32851152327 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:02:52.447161 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c27b7c-76e4-4d87-9885-e32851152327 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:02:52.468387 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-a0c27b7c-76e4-4d87-9885-e32851152327 None None] MFA Rules not processed for user `229e076684dd4bbdacda2231b18f0875`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 15:02:52.469707 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-a0c27b7c-76e4-4d87-9885-e32851152327 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:02:52.527051 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1222/2433] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 15:02:52 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 15:02:59.135252 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=20 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 15:03:02.519613 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=25 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 15:03:04.809406 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4adcd3-6ea3-41c8-a4e7-5d56208e33b6 None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:03:04.809915 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4adcd3-6ea3-41c8-a4e7-5d56208e33b6 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:03:04.810022 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4adcd3-6ea3-41c8-a4e7-5d56208e33b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:03:04.829188 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-cf4adcd3-6ea3-41c8-a4e7-5d56208e33b6 None None] MFA Rules not processed for user `afa6212f31034e27a8d32f114937190e`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 15:03:04.830421 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-cf4adcd3-6ea3-41c8-a4e7-5d56208e33b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:03:04.837947 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1212/2434] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 15:03:04 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 15:03:14.823452 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=3 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 15:03:21.684052 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c27b7c-76e4-4d87-9885-e32851152327 None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:03:21.685615 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-a0c27b7c-76e4-4d87-9885-e32851152327 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:03:21.750105 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c27b7c-76e4-4d87-9885-e32851152327 None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:03:21.754485 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef771765-4613-45d4-be03-ac304402caa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:03:21.755101 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef771765-4613-45d4-be03-ac304402caa9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:03:21.755373 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef771765-4613-45d4-be03-ac304402caa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:03:21.755635 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef771765-4613-45d4-be03-ac304402caa9 admin admin] PATH_INFO: `/v3/users/74b4bf3f69204fc896c996bcaa9e087e` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:03:21.756741 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef771765-4613-45d4-be03-ac304402caa9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=74b4bf3f69204fc896c996bcaa9e087e)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:03:21.759306 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef771765-4613-45d4-be03-ac304402caa9 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:03:21.804560 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-ef771765-4613-45d4-be03-ac304402caa9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '74b4bf3f69204fc896c996bcaa9e087e'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:03:21.817259 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-ef771765-4613-45d4-be03-ac304402caa9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '74b4bf3f69204fc896c996bcaa9e087e'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:03:21.817949 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-ef771765-4613-45d4-be03-ac304402caa9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '74b4bf3f69204fc896c996bcaa9e087e'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:03:21.831118 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-ef771765-4613-45d4-be03-ac304402caa9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '74b4bf3f69204fc896c996bcaa9e087e'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:03:21.831758 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-ef771765-4613-45d4-be03-ac304402caa9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '74b4bf3f69204fc896c996bcaa9e087e'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:03:21.843745 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1223/2435] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Sep 1 15:03:21 2022] DELETE /identity/v3/users/74b4bf3f69204fc896c996bcaa9e087e => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 15:03:22.018775 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4adcd3-6ea3-41c8-a4e7-5d56208e33b6 None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:03:22.031254 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4adcd3-6ea3-41c8-a4e7-5d56208e33b6 None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:03:22.095419 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e0753c-23de-4971-8560-c5ac16ebfe9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:03:22.096198 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e0753c-23de-4971-8560-c5ac16ebfe9d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:03:22.096586 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e0753c-23de-4971-8560-c5ac16ebfe9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:03:22.096964 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e0753c-23de-4971-8560-c5ac16ebfe9d admin admin] PATH_INFO: `/v3/projects/53f5d57b243e421cbf8d929cafe9c860` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:03:22.098343 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e0753c-23de-4971-8560-c5ac16ebfe9d admin admin] RBAC: Authorizing `identity:delete_project(project_id=53f5d57b243e421cbf8d929cafe9c860)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:03:22.100317 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e0753c-23de-4971-8560-c5ac16ebfe9d admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:03:22.147404 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-88e0753c-23de-4971-8560-c5ac16ebfe9d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '53f5d57b243e421cbf8d929cafe9c860', 'request_id': 'req-88e0753c-23de-4971-8560-c5ac16ebfe9d'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:03:22.153536 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-88e0753c-23de-4971-8560-c5ac16ebfe9d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '53f5d57b243e421cbf8d929cafe9c860', 'request_id': 'req-88e0753c-23de-4971-8560-c5ac16ebfe9d'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:03:22.160930 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-88e0753c-23de-4971-8560-c5ac16ebfe9d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '53f5d57b243e421cbf8d929cafe9c860', 'request_id': 'req-88e0753c-23de-4971-8560-c5ac16ebfe9d'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:03:22.171824 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-88e0753c-23de-4971-8560-c5ac16ebfe9d admin admin] The token cache is being invalidate because project 53f5d57b243e421cbf8d929cafe9c860 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81341) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 15:03:22.172107 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-88e0753c-23de-4971-8560-c5ac16ebfe9d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:03:22.172727 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-88e0753c-23de-4971-8560-c5ac16ebfe9d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:03:22.174564 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1213/2436] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Sep 1 15:03:22 2022] DELETE /identity/v3/projects/53f5d57b243e421cbf8d929cafe9c860 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 15:03:22.179445 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb5e7f0-f46a-4fd0-9b04-c92db6fde3b4 None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:03:22.179761 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb5e7f0-f46a-4fd0-9b04-c92db6fde3b4 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:03:22.180033 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb5e7f0-f46a-4fd0-9b04-c92db6fde3b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:03:22.200175 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-cbb5e7f0-f46a-4fd0-9b04-c92db6fde3b4 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 15:03:22.201362 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-cbb5e7f0-f46a-4fd0-9b04-c92db6fde3b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:03:22.259416 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1224/2437] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 15:03:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 15:03:22.267106 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dceb31e1-273c-4f99-a7ce-35db60e8c799 None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:03:22.267332 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dceb31e1-273c-4f99-a7ce-35db60e8c799 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:03:22.267609 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dceb31e1-273c-4f99-a7ce-35db60e8c799 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:03:22.287540 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-dceb31e1-273c-4f99-a7ce-35db60e8c799 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 15:03:22.288742 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-dceb31e1-273c-4f99-a7ce-35db60e8c799 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:03:22.297159 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1214/2438] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 15:03:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 15:03:22.301443 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb5e7f0-f46a-4fd0-9b04-c92db6fde3b4 None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:03:22.311121 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb5e7f0-f46a-4fd0-9b04-c92db6fde3b4 None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:03:22.316396 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34128428-bce6-49ca-87e7-b306fa60122a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:03:22.316985 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34128428-bce6-49ca-87e7-b306fa60122a admin admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:03:22.317258 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34128428-bce6-49ca-87e7-b306fa60122a admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:03:22.317523 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34128428-bce6-49ca-87e7-b306fa60122a admin admin] PATH_INFO: `/v3/domains` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:03:22.318161 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34128428-bce6-49ca-87e7-b306fa60122a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 15:03:22.318487 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34128428-bce6-49ca-87e7-b306fa60122a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:03:22.320458 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34128428-bce6-49ca-87e7-b306fa60122a admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:03:22.331962 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1225/2439] 10.222.0.33 () {58 vars in 1153 bytes} [Thu Sep 1 15:03:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 15:03:22.336330 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dceb31e1-273c-4f99-a7ce-35db60e8c799 None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:03:22.346160 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dceb31e1-273c-4f99-a7ce-35db60e8c799 None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:03:22.350790 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dac5828-9e59-47cf-b6dc-beb4ce9f2aa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:03:22.351371 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dac5828-9e59-47cf-b6dc-beb4ce9f2aa7 admin admin] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:03:22.351649 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dac5828-9e59-47cf-b6dc-beb4ce9f2aa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:03:22.351918 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dac5828-9e59-47cf-b6dc-beb4ce9f2aa7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:03:22.353110 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dac5828-9e59-47cf-b6dc-beb4ce9f2aa7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:03:22.355045 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dac5828-9e59-47cf-b6dc-beb4ce9f2aa7 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:03:22.380908 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1215/2440] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Sep 1 15:03:22 2022] POST /identity/v3/projects => generated 371 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 15:03:22.384991 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34128428-bce6-49ca-87e7-b306fa60122a admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:03:22.394822 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34128428-bce6-49ca-87e7-b306fa60122a admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:03:22.398305 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 15:03:22.452745 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746b417e-ffd5-4a4b-a262-28b3f02f348b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:03:22.453338 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746b417e-ffd5-4a4b-a262-28b3f02f348b admin admin] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:03:22.453619 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746b417e-ffd5-4a4b-a262-28b3f02f348b admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:03:22.453888 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746b417e-ffd5-4a4b-a262-28b3f02f348b admin admin] PATH_INFO: `/v3/users` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:03:22.454554 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746b417e-ffd5-4a4b-a262-28b3f02f348b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:03:22.456463 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746b417e-ffd5-4a4b-a262-28b3f02f348b admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:03:22.476760 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1226/2441] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Sep 1 15:03:22 2022] POST /identity/v3/users => generated 286 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 15:03:22.481513 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dac5828-9e59-47cf-b6dc-beb4ce9f2aa7 admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:03:22.491698 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dac5828-9e59-47cf-b6dc-beb4ce9f2aa7 admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:03:22.496343 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de13eaa2-fc0d-4c1a-9814-7c0d4485d384 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:03:22.496932 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de13eaa2-fc0d-4c1a-9814-7c0d4485d384 admin admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:03:22.497301 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de13eaa2-fc0d-4c1a-9814-7c0d4485d384 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:03:22.497574 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de13eaa2-fc0d-4c1a-9814-7c0d4485d384 admin admin] PATH_INFO: `/v3/roles` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:03:22.498213 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de13eaa2-fc0d-4c1a-9814-7c0d4485d384 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:03:22.499845 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de13eaa2-fc0d-4c1a-9814-7c0d4485d384 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:03:22.509569 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1216/2442] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 1 15:03:22 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 15:03:22.513403 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746b417e-ffd5-4a4b-a262-28b3f02f348b admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:03:22.523213 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746b417e-ffd5-4a4b-a262-28b3f02f348b admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:03:22.528260 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5e41d4-aa19-4c4a-94a9-23608f3372c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:03:22.528843 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5e41d4-aa19-4c4a-94a9-23608f3372c8 admin admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:03:22.529122 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5e41d4-aa19-4c4a-94a9-23608f3372c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:03:22.529393 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5e41d4-aa19-4c4a-94a9-23608f3372c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:03:22.530027 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5e41d4-aa19-4c4a-94a9-23608f3372c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:03:22.531652 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5e41d4-aa19-4c4a-94a9-23608f3372c8 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:03:22.542506 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1227/2443] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 1 15:03:22 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 15:03:22.546317 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de13eaa2-fc0d-4c1a-9814-7c0d4485d384 admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:03:22.556296 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de13eaa2-fc0d-4c1a-9814-7c0d4485d384 admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:03:22.560924 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e631a119-7eca-4166-8f0f-920c496e7157 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:03:22.561558 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e631a119-7eca-4166-8f0f-920c496e7157 admin admin] REQUEST_METHOD: `PUT` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:03:22.561837 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e631a119-7eca-4166-8f0f-920c496e7157 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:03:22.562108 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e631a119-7eca-4166-8f0f-920c496e7157 admin admin] PATH_INFO: `/v3/projects/764329f7a06b42969616baffd95cbd29/users/210bc3fc36724b32ba50364332c6f900/roles/5d1feef0e13d4874928bd86258c4e3b2` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:03:22.577605 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e631a119-7eca-4166-8f0f-920c496e7157 admin admin] RBAC: Authorizing `identity:create_grant(project_id=764329f7a06b42969616baffd95cbd29, user_id=210bc3fc36724b32ba50364332c6f900, role_id=5d1feef0e13d4874928bd86258c4e3b2)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:03:22.580291 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e631a119-7eca-4166-8f0f-920c496e7157 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:03:22.593431 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1217/2444] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 1 15:03:22 2022] PUT /identity/v3/projects/764329f7a06b42969616baffd95cbd29/users/210bc3fc36724b32ba50364332c6f900/roles/5d1feef0e13d4874928bd86258c4e3b2 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 15:03:22.598621 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c9ec09-539c-4d1e-b061-e39bac9a4153 None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:03:22.598922 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c9ec09-539c-4d1e-b061-e39bac9a4153 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:03:22.599174 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c9ec09-539c-4d1e-b061-e39bac9a4153 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:03:22.617555 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-f0c9ec09-539c-4d1e-b061-e39bac9a4153 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 15:03:22.618802 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-f0c9ec09-539c-4d1e-b061-e39bac9a4153 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:03:22.677521 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1228/2445] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 15:03:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 15:03:22.687277 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e631a119-7eca-4166-8f0f-920c496e7157 admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:03:22.689017 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-e631a119-7eca-4166-8f0f-920c496e7157 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:03:22.722226 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 15:03:22.750818 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e631a119-7eca-4166-8f0f-920c496e7157 admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:03:22.754408 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849f5660-df0f-4a44-bc88-f58db26d1d4e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2790149639aa418bbc4e4982e93eda44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:03:22.755005 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849f5660-df0f-4a44-bc88-f58db26d1d4e service neutron] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:03:22.755282 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849f5660-df0f-4a44-bc88-f58db26d1d4e service neutron] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:03:22.755549 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849f5660-df0f-4a44-bc88-f58db26d1d4e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:03:22.764847 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849f5660-df0f-4a44-bc88-f58db26d1d4e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:03:22.766320 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849f5660-df0f-4a44-bc88-f58db26d1d4e service neutron] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:03:22.776588 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1218/2446] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 15:03:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 15:03:24.490867 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c9ec09-539c-4d1e-b061-e39bac9a4153 None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:03:24.492331 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-f0c9ec09-539c-4d1e-b061-e39bac9a4153 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:03:24.509220 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c9ec09-539c-4d1e-b061-e39bac9a4153 None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:03:24.513375 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96df6765-beb5-4517-ae41-682f5a7ff8df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2790149639aa418bbc4e4982e93eda44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:03:24.513993 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96df6765-beb5-4517-ae41-682f5a7ff8df service neutron] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:03:24.514379 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96df6765-beb5-4517-ae41-682f5a7ff8df service neutron] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:03:24.514932 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96df6765-beb5-4517-ae41-682f5a7ff8df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:03:24.516741 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-96df6765-beb5-4517-ae41-682f5a7ff8df service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:03:24.598875 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96df6765-beb5-4517-ae41-682f5a7ff8df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:03:24.600290 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96df6765-beb5-4517-ae41-682f5a7ff8df service neutron] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:03:24.612716 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1229/2447] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 15:03:24 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 15:03:26.081049 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849f5660-df0f-4a44-bc88-f58db26d1d4e service neutron] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:03:26.094114 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849f5660-df0f-4a44-bc88-f58db26d1d4e service neutron] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:03:26.098346 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63497e5e-11ea-41ca-8a1c-0872103f7cf6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2790149639aa418bbc4e4982e93eda44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:03:26.099254 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63497e5e-11ea-41ca-8a1c-0872103f7cf6 service neutron] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:03:26.099739 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63497e5e-11ea-41ca-8a1c-0872103f7cf6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:03:26.100273 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63497e5e-11ea-41ca-8a1c-0872103f7cf6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:03:26.102808 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-63497e5e-11ea-41ca-8a1c-0872103f7cf6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:03:26.128603 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63497e5e-11ea-41ca-8a1c-0872103f7cf6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:03:26.130227 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63497e5e-11ea-41ca-8a1c-0872103f7cf6 service neutron] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:03:26.160525 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1219/2448] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 15:03:26 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 15:03:26.695695 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96df6765-beb5-4517-ae41-682f5a7ff8df service neutron] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:03:26.705814 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96df6765-beb5-4517-ae41-682f5a7ff8df service neutron] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:03:26.709826 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13de2dfc-de67-4474-8080-65c549694b32 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2790149639aa418bbc4e4982e93eda44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:03:26.711220 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13de2dfc-de67-4474-8080-65c549694b32 service neutron] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:03:26.711820 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13de2dfc-de67-4474-8080-65c549694b32 service neutron] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:03:26.712088 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13de2dfc-de67-4474-8080-65c549694b32 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:03:26.715062 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-13de2dfc-de67-4474-8080-65c549694b32 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:03:26.740952 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 15:03:26.962809 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13de2dfc-de67-4474-8080-65c549694b32 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:03:26.966227 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13de2dfc-de67-4474-8080-65c549694b32 service neutron] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:03:26.985285 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1230/2449] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 15:03:26 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 15:03:27.148692 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb3862b-3c1f-4098-8e3b-93cf919d5d4e None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:03:27.149268 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb3862b-3c1f-4098-8e3b-93cf919d5d4e None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:03:27.149389 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb3862b-3c1f-4098-8e3b-93cf919d5d4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:03:27.181360 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-7cb3862b-3c1f-4098-8e3b-93cf919d5d4e None None] MFA Rules not processed for user `210bc3fc36724b32ba50364332c6f900`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 15:03:27.182743 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-7cb3862b-3c1f-4098-8e3b-93cf919d5d4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:03:27.257267 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1220/2450] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 15:03:27 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 15:03:27.261798 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13de2dfc-de67-4474-8080-65c549694b32 service neutron] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:03:27.273211 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13de2dfc-de67-4474-8080-65c549694b32 service neutron] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:03:27.278568 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0b1734-de02-464f-8e21-07fb6808295b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:03:27.279412 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0b1734-de02-464f-8e21-07fb6808295b admin admin] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:03:27.279687 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0b1734-de02-464f-8e21-07fb6808295b admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:03:27.279954 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0b1734-de02-464f-8e21-07fb6808295b admin admin] PATH_INFO: `/v3/projects` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:03:27.280590 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0b1734-de02-464f-8e21-07fb6808295b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:03:27.282759 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0b1734-de02-464f-8e21-07fb6808295b admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:03:27.320462 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1231/2451] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Sep 1 15:03:27 2022] POST /identity/v3/projects => generated 373 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 15:03:27.321675 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb3862b-3c1f-4098-8e3b-93cf919d5d4e None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:03:27.333641 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb3862b-3c1f-4098-8e3b-93cf919d5d4e None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:03:27.404589 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f296478a-1a03-4256-bd50-85fe347fc681 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:03:27.405652 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f296478a-1a03-4256-bd50-85fe347fc681 admin admin] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:03:27.405652 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f296478a-1a03-4256-bd50-85fe347fc681 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:03:27.405883 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f296478a-1a03-4256-bd50-85fe347fc681 admin admin] PATH_INFO: `/v3/users` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:03:27.406741 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f296478a-1a03-4256-bd50-85fe347fc681 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:03:27.408786 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f296478a-1a03-4256-bd50-85fe347fc681 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:03:27.436794 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1221/2452] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Sep 1 15:03:27 2022] POST /identity/v3/users => generated 293 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 15:03:27.440789 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0b1734-de02-464f-8e21-07fb6808295b admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:03:27.451911 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0b1734-de02-464f-8e21-07fb6808295b admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:03:27.461425 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fccbb3-077e-4ba4-b145-aa5c4ace93e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:03:27.462644 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fccbb3-077e-4ba4-b145-aa5c4ace93e8 admin admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:03:27.463052 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fccbb3-077e-4ba4-b145-aa5c4ace93e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:03:27.463180 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fccbb3-077e-4ba4-b145-aa5c4ace93e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:03:27.463830 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fccbb3-077e-4ba4-b145-aa5c4ace93e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:03:27.465717 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fccbb3-077e-4ba4-b145-aa5c4ace93e8 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:03:27.475747 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1232/2453] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 1 15:03:27 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 15:03:27.480664 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f296478a-1a03-4256-bd50-85fe347fc681 admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:03:27.492249 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f296478a-1a03-4256-bd50-85fe347fc681 admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:03:27.496969 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196dd097-1a0e-43cf-8aa5-cfdf8ca0b4ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:03:27.497616 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196dd097-1a0e-43cf-8aa5-cfdf8ca0b4ec admin admin] REQUEST_METHOD: `PUT` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:03:27.497896 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196dd097-1a0e-43cf-8aa5-cfdf8ca0b4ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:03:27.498164 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196dd097-1a0e-43cf-8aa5-cfdf8ca0b4ec admin admin] PATH_INFO: `/v3/projects/0063eabaf1ec4cd5bd120ca3db0622af/users/ebda997302e24adb962d50bad531ed63/roles/2a2adb769e55476b82e5014897e0a9c0` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:03:27.513194 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196dd097-1a0e-43cf-8aa5-cfdf8ca0b4ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=0063eabaf1ec4cd5bd120ca3db0622af, user_id=ebda997302e24adb962d50bad531ed63, role_id=2a2adb769e55476b82e5014897e0a9c0)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:03:27.515895 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196dd097-1a0e-43cf-8aa5-cfdf8ca0b4ec admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:03:27.531001 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1222/2454] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 1 15:03:27 2022] PUT /identity/v3/projects/0063eabaf1ec4cd5bd120ca3db0622af/users/ebda997302e24adb962d50bad531ed63/roles/2a2adb769e55476b82e5014897e0a9c0 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 15:03:27.534996 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fccbb3-077e-4ba4-b145-aa5c4ace93e8 admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:03:27.546829 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fccbb3-077e-4ba4-b145-aa5c4ace93e8 admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:03:27.605371 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5153b4-eb95-4c95-abe7-b2eec78fc9e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:03:27.605899 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5153b4-eb95-4c95-abe7-b2eec78fc9e7 admin admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:03:27.606180 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5153b4-eb95-4c95-abe7-b2eec78fc9e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:03:27.606493 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5153b4-eb95-4c95-abe7-b2eec78fc9e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:03:27.607138 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5153b4-eb95-4c95-abe7-b2eec78fc9e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:03:27.608741 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5153b4-eb95-4c95-abe7-b2eec78fc9e7 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:03:27.619119 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1233/2455] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 1 15:03:27 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 15:03:27.622950 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196dd097-1a0e-43cf-8aa5-cfdf8ca0b4ec admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:03:27.633916 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196dd097-1a0e-43cf-8aa5-cfdf8ca0b4ec admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:03:27.638756 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c74ed97-6f09-4954-8d70-dc6a9288f233 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:03:27.639477 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c74ed97-6f09-4954-8d70-dc6a9288f233 admin admin] REQUEST_METHOD: `PUT` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:03:27.639850 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c74ed97-6f09-4954-8d70-dc6a9288f233 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:03:27.640294 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c74ed97-6f09-4954-8d70-dc6a9288f233 admin admin] PATH_INFO: `/v3/projects/0063eabaf1ec4cd5bd120ca3db0622af/users/ebda997302e24adb962d50bad531ed63/roles/2a2adb769e55476b82e5014897e0a9c0` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:03:27.642399 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c74ed97-6f09-4954-8d70-dc6a9288f233 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0063eabaf1ec4cd5bd120ca3db0622af, user_id=ebda997302e24adb962d50bad531ed63, role_id=2a2adb769e55476b82e5014897e0a9c0)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:03:27.646410 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c74ed97-6f09-4954-8d70-dc6a9288f233 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:03:27.654296 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 15:03:27.663428 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1223/2456] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 1 15:03:27 2022] PUT /identity/v3/projects/0063eabaf1ec4cd5bd120ca3db0622af/users/ebda997302e24adb962d50bad531ed63/roles/2a2adb769e55476b82e5014897e0a9c0 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 15:03:32.374090 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfd59c2-4d9a-4c31-9742-6f0b62dd87b2 None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:03:32.375082 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfd59c2-4d9a-4c31-9742-6f0b62dd87b2 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:03:32.375520 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfd59c2-4d9a-4c31-9742-6f0b62dd87b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:03:32.433551 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-ebfd59c2-4d9a-4c31-9742-6f0b62dd87b2 None None] MFA Rules not processed for user `ebda997302e24adb962d50bad531ed63`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 15:03:32.434972 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-ebfd59c2-4d9a-4c31-9742-6f0b62dd87b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:03:32.520876 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 15:03:32.535049 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1234/2457] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 15:03:32 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 15:03:32.552414 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c74ed97-6f09-4954-8d70-dc6a9288f233 admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:03:32.554154 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-6c74ed97-6f09-4954-8d70-dc6a9288f233 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:03:32.668068 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c74ed97-6f09-4954-8d70-dc6a9288f233 admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:03:32.674293 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fab95b8-999b-469f-98e8-c75380de39d7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4858bb8621854ec8aba836e7d9050498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:03:32.674909 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fab95b8-999b-469f-98e8-c75380de39d7 service nova] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:03:32.675187 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fab95b8-999b-469f-98e8-c75380de39d7 service nova] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:03:32.675453 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fab95b8-999b-469f-98e8-c75380de39d7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:03:32.748338 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fab95b8-999b-469f-98e8-c75380de39d7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:03:32.751424 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fab95b8-999b-469f-98e8-c75380de39d7 service nova] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:03:32.767718 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1224/2458] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 15:03:32 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 15:03:33.853327 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056d1d94-6004-4ae4-9219-a20f6281eb22 None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:03:33.853844 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056d1d94-6004-4ae4-9219-a20f6281eb22 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:03:33.853948 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056d1d94-6004-4ae4-9219-a20f6281eb22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:03:33.872423 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-056d1d94-6004-4ae4-9219-a20f6281eb22 None None] MFA Rules not processed for user `210bc3fc36724b32ba50364332c6f900`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 15:03:33.873660 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-056d1d94-6004-4ae4-9219-a20f6281eb22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:03:33.880579 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1235/2459] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 15:03:33 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 15:03:36.719534 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fab95b8-999b-469f-98e8-c75380de39d7 service nova] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:03:36.731251 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fab95b8-999b-469f-98e8-c75380de39d7 service nova] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:03:36.736587 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e36c26-bae7-4ffb-8641-1cb0229eaa86 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4858bb8621854ec8aba836e7d9050498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:03:36.737168 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e36c26-bae7-4ffb-8641-1cb0229eaa86 service nova] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:03:36.737443 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e36c26-bae7-4ffb-8641-1cb0229eaa86 service nova] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:03:36.737711 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e36c26-bae7-4ffb-8641-1cb0229eaa86 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:03:36.739427 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-62e36c26-bae7-4ffb-8641-1cb0229eaa86 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:03:36.798158 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e36c26-bae7-4ffb-8641-1cb0229eaa86 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:03:36.799829 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e36c26-bae7-4ffb-8641-1cb0229eaa86 service nova] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:03:36.809391 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1225/2460] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 15:03:36 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 15:03:38.016428 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056d1d94-6004-4ae4-9219-a20f6281eb22 None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:03:38.017941 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-056d1d94-6004-4ae4-9219-a20f6281eb22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:03:38.034764 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056d1d94-6004-4ae4-9219-a20f6281eb22 None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:03:38.040145 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ace1b6c-8247-4fec-a383-b9f051cdc84f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4858bb8621854ec8aba836e7d9050498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:03:38.041466 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ace1b6c-8247-4fec-a383-b9f051cdc84f service nova] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:03:38.041833 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ace1b6c-8247-4fec-a383-b9f051cdc84f service nova] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:03:38.042152 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ace1b6c-8247-4fec-a383-b9f051cdc84f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:03:38.044006 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-4ace1b6c-8247-4fec-a383-b9f051cdc84f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:03:38.109570 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ace1b6c-8247-4fec-a383-b9f051cdc84f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:03:38.111124 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ace1b6c-8247-4fec-a383-b9f051cdc84f service nova] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:03:38.120655 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1236/2461] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 15:03:38 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 15:03:45.265817 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cac32f-bc30-444b-a6d8-642aa089c308 None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:03:45.266505 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cac32f-bc30-444b-a6d8-642aa089c308 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:03:45.266895 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cac32f-bc30-444b-a6d8-642aa089c308 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:03:45.287743 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-37cac32f-bc30-444b-a6d8-642aa089c308 None None] MFA Rules not processed for user `82255a5060674d77a554e34f89ab1bff`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 15:03:45.288960 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-37cac32f-bc30-444b-a6d8-642aa089c308 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:03:45.296129 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1226/2462] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 15:03:45 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 15:03:45.608454 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ace1b6c-8247-4fec-a383-b9f051cdc84f service nova] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:03:45.671825 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ace1b6c-8247-4fec-a383-b9f051cdc84f service nova] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:03:45.675686 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a274c3d2-5806-49ea-897a-cd275cf120f2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2790149639aa418bbc4e4982e93eda44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:03:45.676295 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a274c3d2-5806-49ea-897a-cd275cf120f2 service neutron] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:03:45.676576 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a274c3d2-5806-49ea-897a-cd275cf120f2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:03:45.676845 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a274c3d2-5806-49ea-897a-cd275cf120f2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:03:45.678630 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-a274c3d2-5806-49ea-897a-cd275cf120f2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:03:45.689177 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a274c3d2-5806-49ea-897a-cd275cf120f2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:03:45.690659 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a274c3d2-5806-49ea-897a-cd275cf120f2 service neutron] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:03:45.699607 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1237/2463] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 15:03:45 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 15:03:46.018143 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cac32f-bc30-444b-a6d8-642aa089c308 None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:03:46.019921 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-37cac32f-bc30-444b-a6d8-642aa089c308 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:03:46.065248 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 15:03:46.127619 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cac32f-bc30-444b-a6d8-642aa089c308 None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:03:46.138464 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfe3d58-6cee-44bb-a9c6-8cd021dd4c1d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f312c8dcabe48ef9ea453364ade38ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:03:46.138464 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfe3d58-6cee-44bb-a9c6-8cd021dd4c1d service placement] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:03:46.138464 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfe3d58-6cee-44bb-a9c6-8cd021dd4c1d service placement] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:03:46.138464 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfe3d58-6cee-44bb-a9c6-8cd021dd4c1d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:03:46.138844 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-6cfe3d58-6cee-44bb-a9c6-8cd021dd4c1d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:03:46.151682 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cfe3d58-6cee-44bb-a9c6-8cd021dd4c1d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:03:46.153526 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cfe3d58-6cee-44bb-a9c6-8cd021dd4c1d service placement] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:03:46.164600 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1227/2464] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 15:03:46 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 15:03:50.699014 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a274c3d2-5806-49ea-897a-cd275cf120f2 service neutron] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:03:50.709130 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a274c3d2-5806-49ea-897a-cd275cf120f2 service neutron] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:03:50.714419 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61f609b-084c-480d-a9e5-27560f3b7bdb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2790149639aa418bbc4e4982e93eda44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:03:50.715260 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61f609b-084c-480d-a9e5-27560f3b7bdb service neutron] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:03:50.715756 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61f609b-084c-480d-a9e5-27560f3b7bdb service neutron] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:03:50.716753 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61f609b-084c-480d-a9e5-27560f3b7bdb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:03:50.719853 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-c61f609b-084c-480d-a9e5-27560f3b7bdb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:03:50.732205 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61f609b-084c-480d-a9e5-27560f3b7bdb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:03:50.734095 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61f609b-084c-480d-a9e5-27560f3b7bdb service neutron] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:03:50.757760 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1238/2465] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 15:03:50 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 15:03:50.904642 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfe3d58-6cee-44bb-a9c6-8cd021dd4c1d service placement] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:03:50.918323 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfe3d58-6cee-44bb-a9c6-8cd021dd4c1d service placement] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:03:50.922518 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5338ddd7-b62a-4c9e-ab20-3d8fcce11d07 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2790149639aa418bbc4e4982e93eda44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:03:50.923600 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5338ddd7-b62a-4c9e-ab20-3d8fcce11d07 service neutron] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:03:50.924030 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5338ddd7-b62a-4c9e-ab20-3d8fcce11d07 service neutron] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:03:50.925185 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5338ddd7-b62a-4c9e-ab20-3d8fcce11d07 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:03:50.927595 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-5338ddd7-b62a-4c9e-ab20-3d8fcce11d07 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:03:50.941050 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5338ddd7-b62a-4c9e-ab20-3d8fcce11d07 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:03:50.943498 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5338ddd7-b62a-4c9e-ab20-3d8fcce11d07 service neutron] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:03:50.955206 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1228/2466] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 15:03:50 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 15:03:57.103180 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61f609b-084c-480d-a9e5-27560f3b7bdb service neutron] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:03:57.120806 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61f609b-084c-480d-a9e5-27560f3b7bdb service neutron] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:03:57.127828 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3faaa4cb-9f3c-40ca-be6f-4669c2627a52 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4858bb8621854ec8aba836e7d9050498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:03:57.128446 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3faaa4cb-9f3c-40ca-be6f-4669c2627a52 service nova] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:03:57.128724 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3faaa4cb-9f3c-40ca-be6f-4669c2627a52 service nova] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:03:57.128991 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3faaa4cb-9f3c-40ca-be6f-4669c2627a52 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:03:57.131343 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-3faaa4cb-9f3c-40ca-be6f-4669c2627a52 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:03:57.201234 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3faaa4cb-9f3c-40ca-be6f-4669c2627a52 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:03:57.202875 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3faaa4cb-9f3c-40ca-be6f-4669c2627a52 service nova] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:03:57.212030 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1239/2467] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 15:03:57 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 15:03:58.185895 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5338ddd7-b62a-4c9e-ab20-3d8fcce11d07 service neutron] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:03:58.194660 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5338ddd7-b62a-4c9e-ab20-3d8fcce11d07 service neutron] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:03:58.199025 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ef2aa0-30a6-4bfb-8da9-85add15012ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2790149639aa418bbc4e4982e93eda44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:03:58.199623 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ef2aa0-30a6-4bfb-8da9-85add15012ef service neutron] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:03:58.199900 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ef2aa0-30a6-4bfb-8da9-85add15012ef service neutron] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:03:58.200307 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ef2aa0-30a6-4bfb-8da9-85add15012ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:03:58.202036 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-b0ef2aa0-30a6-4bfb-8da9-85add15012ef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:03:58.268742 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ef2aa0-30a6-4bfb-8da9-85add15012ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:03:58.270210 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ef2aa0-30a6-4bfb-8da9-85add15012ef service neutron] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:03:58.280543 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1229/2468] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 15:03:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 15:04:01.800901 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3faaa4cb-9f3c-40ca-be6f-4669c2627a52 service nova] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:04:01.802427 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-3faaa4cb-9f3c-40ca-be6f-4669c2627a52 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:04:01.818084 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3faaa4cb-9f3c-40ca-be6f-4669c2627a52 service nova] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:04:01.822885 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf79ed7a-9c34-4106-82bb-9f6cf25449a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:04:01.823510 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf79ed7a-9c34-4106-82bb-9f6cf25449a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:04:01.823783 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf79ed7a-9c34-4106-82bb-9f6cf25449a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:04:01.824310 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf79ed7a-9c34-4106-82bb-9f6cf25449a8 admin admin] PATH_INFO: `/v3/users/df75151c37f74b6da7428dc290a274ff` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:04:01.825446 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf79ed7a-9c34-4106-82bb-9f6cf25449a8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=df75151c37f74b6da7428dc290a274ff)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:04:01.827367 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf79ed7a-9c34-4106-82bb-9f6cf25449a8 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:04:01.832833 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 15:04:01.876096 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-cf79ed7a-9c34-4106-82bb-9f6cf25449a8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'df75151c37f74b6da7428dc290a274ff'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:04:01.884497 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-cf79ed7a-9c34-4106-82bb-9f6cf25449a8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'df75151c37f74b6da7428dc290a274ff'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:04:01.885367 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-cf79ed7a-9c34-4106-82bb-9f6cf25449a8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'df75151c37f74b6da7428dc290a274ff'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:04:01.897382 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-cf79ed7a-9c34-4106-82bb-9f6cf25449a8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'df75151c37f74b6da7428dc290a274ff'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:04:01.898009 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-cf79ed7a-9c34-4106-82bb-9f6cf25449a8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'df75151c37f74b6da7428dc290a274ff'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:04:01.910475 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1240/2469] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Sep 1 15:04:01 2022] DELETE /identity/v3/users/df75151c37f74b6da7428dc290a274ff => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 15:04:02.097600 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ef2aa0-30a6-4bfb-8da9-85add15012ef service neutron] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:04:02.112818 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ef2aa0-30a6-4bfb-8da9-85add15012ef service neutron] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:04:02.192326 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f625d964-7690-4963-aad4-49d56ffdafe3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:04:02.193192 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f625d964-7690-4963-aad4-49d56ffdafe3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:04:02.193473 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f625d964-7690-4963-aad4-49d56ffdafe3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:04:02.193745 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f625d964-7690-4963-aad4-49d56ffdafe3 admin admin] PATH_INFO: `/v3/projects/05e111f88e784eba929741c3cd250e78` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:04:02.194924 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f625d964-7690-4963-aad4-49d56ffdafe3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=05e111f88e784eba929741c3cd250e78)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:04:02.197050 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f625d964-7690-4963-aad4-49d56ffdafe3 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:04:02.237176 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 15:04:02.249440 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-f625d964-7690-4963-aad4-49d56ffdafe3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '05e111f88e784eba929741c3cd250e78', 'request_id': 'req-f625d964-7690-4963-aad4-49d56ffdafe3'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:04:02.255737 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-f625d964-7690-4963-aad4-49d56ffdafe3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '05e111f88e784eba929741c3cd250e78', 'request_id': 'req-f625d964-7690-4963-aad4-49d56ffdafe3'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:04:02.263450 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-f625d964-7690-4963-aad4-49d56ffdafe3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '05e111f88e784eba929741c3cd250e78', 'request_id': 'req-f625d964-7690-4963-aad4-49d56ffdafe3'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:04:02.277309 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-f625d964-7690-4963-aad4-49d56ffdafe3 admin admin] The token cache is being invalidate because project 05e111f88e784eba929741c3cd250e78 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81341) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 15:04:02.277691 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-f625d964-7690-4963-aad4-49d56ffdafe3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:04:02.278378 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-f625d964-7690-4963-aad4-49d56ffdafe3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:04:02.279883 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1230/2470] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Sep 1 15:04:02 2022] DELETE /identity/v3/projects/05e111f88e784eba929741c3cd250e78 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 15:04:02.285045 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a04f433-f697-4a68-8dd8-1d754d59d121 None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:04:02.285326 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a04f433-f697-4a68-8dd8-1d754d59d121 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:04:02.285594 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a04f433-f697-4a68-8dd8-1d754d59d121 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:04:02.306553 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-3a04f433-f697-4a68-8dd8-1d754d59d121 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 15:04:02.307776 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-3a04f433-f697-4a68-8dd8-1d754d59d121 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:04:02.367952 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1241/2471] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 15:04:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 15:04:02.375821 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c47e4e-470c-4f7d-8db9-dd9d095a0aea None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:04:02.376170 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c47e4e-470c-4f7d-8db9-dd9d095a0aea None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:04:02.376509 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c47e4e-470c-4f7d-8db9-dd9d095a0aea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:04:02.400593 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-79c47e4e-470c-4f7d-8db9-dd9d095a0aea None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 15:04:02.401805 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-79c47e4e-470c-4f7d-8db9-dd9d095a0aea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:04:02.409725 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1231/2472] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 15:04:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 15:04:02.413904 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a04f433-f697-4a68-8dd8-1d754d59d121 None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:04:02.423765 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a04f433-f697-4a68-8dd8-1d754d59d121 None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:04:02.428416 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c17b986-61f5-478e-82f1-51dadfbe7310 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:04:02.429008 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c17b986-61f5-478e-82f1-51dadfbe7310 admin admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:04:02.429281 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c17b986-61f5-478e-82f1-51dadfbe7310 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:04:02.429545 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c17b986-61f5-478e-82f1-51dadfbe7310 admin admin] PATH_INFO: `/v3/domains` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:04:02.430181 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c17b986-61f5-478e-82f1-51dadfbe7310 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 15:04:02.430574 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c17b986-61f5-478e-82f1-51dadfbe7310 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:04:02.432137 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c17b986-61f5-478e-82f1-51dadfbe7310 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:04:02.443351 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1242/2473] 10.222.0.33 () {58 vars in 1153 bytes} [Thu Sep 1 15:04:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 15:04:02.447913 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c47e4e-470c-4f7d-8db9-dd9d095a0aea None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:04:02.456991 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c47e4e-470c-4f7d-8db9-dd9d095a0aea None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:04:02.461727 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a439157-843a-4984-93c1-baea91e0822a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:04:02.462858 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a439157-843a-4984-93c1-baea91e0822a admin admin] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:04:02.463137 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a439157-843a-4984-93c1-baea91e0822a admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:04:02.463406 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a439157-843a-4984-93c1-baea91e0822a admin admin] PATH_INFO: `/v3/projects` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:04:02.464027 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a439157-843a-4984-93c1-baea91e0822a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:04:02.465940 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a439157-843a-4984-93c1-baea91e0822a admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:04:02.492070 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1232/2474] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Sep 1 15:04:02 2022] POST /identity/v3/projects => generated 385 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 15:04:02.496087 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c17b986-61f5-478e-82f1-51dadfbe7310 admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:04:02.507372 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c17b986-61f5-478e-82f1-51dadfbe7310 admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:04:02.567236 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1922c5-91be-40c5-b192-7c890b9cd238 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:04:02.567985 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1922c5-91be-40c5-b192-7c890b9cd238 admin admin] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:04:02.568265 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1922c5-91be-40c5-b192-7c890b9cd238 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:04:02.568530 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1922c5-91be-40c5-b192-7c890b9cd238 admin admin] PATH_INFO: `/v3/users` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:04:02.569175 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1922c5-91be-40c5-b192-7c890b9cd238 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:04:02.571642 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1922c5-91be-40c5-b192-7c890b9cd238 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:04:02.595868 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1243/2475] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Sep 1 15:04:02 2022] POST /identity/v3/users => generated 293 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 15:04:02.602921 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a439157-843a-4984-93c1-baea91e0822a admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:04:02.616849 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a439157-843a-4984-93c1-baea91e0822a admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:04:02.621446 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7136bbda-7275-4daf-bc6b-bf4814b3afd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:04:02.622038 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7136bbda-7275-4daf-bc6b-bf4814b3afd8 admin admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:04:02.622353 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7136bbda-7275-4daf-bc6b-bf4814b3afd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:04:02.622621 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7136bbda-7275-4daf-bc6b-bf4814b3afd8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:04:02.623259 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7136bbda-7275-4daf-bc6b-bf4814b3afd8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:04:02.624916 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7136bbda-7275-4daf-bc6b-bf4814b3afd8 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:04:02.635747 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1233/2476] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 1 15:04:02 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 15:04:02.639572 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1922c5-91be-40c5-b192-7c890b9cd238 admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:04:02.644648 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 15:04:02.651901 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1922c5-91be-40c5-b192-7c890b9cd238 admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:04:02.656394 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d5908d-8db4-4c8e-a2c4-2f6fefecf6a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:04:02.657193 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d5908d-8db4-4c8e-a2c4-2f6fefecf6a8 admin admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:04:02.657822 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d5908d-8db4-4c8e-a2c4-2f6fefecf6a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:04:02.658098 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d5908d-8db4-4c8e-a2c4-2f6fefecf6a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:04:02.658919 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d5908d-8db4-4c8e-a2c4-2f6fefecf6a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:04:02.660854 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d5908d-8db4-4c8e-a2c4-2f6fefecf6a8 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:04:02.674318 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1244/2477] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 1 15:04:02 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 15:04:02.679164 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7136bbda-7275-4daf-bc6b-bf4814b3afd8 admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:04:02.690029 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7136bbda-7275-4daf-bc6b-bf4814b3afd8 admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:04:02.694685 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ceca93-1178-421c-a431-09a17cd8b959 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:04:02.695364 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ceca93-1178-421c-a431-09a17cd8b959 admin admin] REQUEST_METHOD: `PUT` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:04:02.695753 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ceca93-1178-421c-a431-09a17cd8b959 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:04:02.696654 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ceca93-1178-421c-a431-09a17cd8b959 admin admin] PATH_INFO: `/v3/projects/0b789cb75bc24a898888246a7745537e/users/27653498cd0a4e089f1cbc43470182ee/roles/5d1feef0e13d4874928bd86258c4e3b2` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:04:02.710975 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ceca93-1178-421c-a431-09a17cd8b959 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b789cb75bc24a898888246a7745537e, user_id=27653498cd0a4e089f1cbc43470182ee, role_id=5d1feef0e13d4874928bd86258c4e3b2)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:04:02.713699 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ceca93-1178-421c-a431-09a17cd8b959 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:04:02.726299 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1234/2478] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 1 15:04:02 2022] PUT /identity/v3/projects/0b789cb75bc24a898888246a7745537e/users/27653498cd0a4e089f1cbc43470182ee/roles/5d1feef0e13d4874928bd86258c4e3b2 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 15:04:02.731274 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8403f7-da72-47d8-b11b-6e72a439a42f None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:04:02.731562 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8403f7-da72-47d8-b11b-6e72a439a42f None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:04:02.731835 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8403f7-da72-47d8-b11b-6e72a439a42f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:04:02.750822 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-fd8403f7-da72-47d8-b11b-6e72a439a42f None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 15:04:02.751966 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-fd8403f7-da72-47d8-b11b-6e72a439a42f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:04:02.808999 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1245/2479] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 15:04:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 15:04:02.818763 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ceca93-1178-421c-a431-09a17cd8b959 admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:04:02.820452 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-e8ceca93-1178-421c-a431-09a17cd8b959 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:04:02.878398 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ceca93-1178-421c-a431-09a17cd8b959 admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:04:02.882085 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7359d972-1f06-4fc6-b34a-fc8d94bea4d4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2790149639aa418bbc4e4982e93eda44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:04:02.882730 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7359d972-1f06-4fc6-b34a-fc8d94bea4d4 service neutron] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:04:02.883006 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7359d972-1f06-4fc6-b34a-fc8d94bea4d4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:04:02.883278 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7359d972-1f06-4fc6-b34a-fc8d94bea4d4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:04:02.893407 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7359d972-1f06-4fc6-b34a-fc8d94bea4d4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:04:02.894907 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7359d972-1f06-4fc6-b34a-fc8d94bea4d4 service neutron] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:04:02.906695 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1235/2480] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 15:04:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 15:04:05.196681 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8403f7-da72-47d8-b11b-6e72a439a42f None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:04:05.198587 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-fd8403f7-da72-47d8-b11b-6e72a439a42f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:04:05.289319 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8403f7-da72-47d8-b11b-6e72a439a42f None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:04:05.298471 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca5f9e7-7237-4d57-a44f-afa3d0a61809 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4858bb8621854ec8aba836e7d9050498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:04:05.299152 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca5f9e7-7237-4d57-a44f-afa3d0a61809 service nova] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:04:05.299432 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca5f9e7-7237-4d57-a44f-afa3d0a61809 service nova] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:04:05.299700 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca5f9e7-7237-4d57-a44f-afa3d0a61809 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:04:05.301559 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-eca5f9e7-7237-4d57-a44f-afa3d0a61809 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:04:05.318729 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca5f9e7-7237-4d57-a44f-afa3d0a61809 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:04:05.321114 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca5f9e7-7237-4d57-a44f-afa3d0a61809 service nova] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:04:05.346968 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1246/2481] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 15:04:05 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 15:04:07.897353 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf424df-07c6-4a8a-ba23-90feca9c38d4 None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:04:07.897884 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf424df-07c6-4a8a-ba23-90feca9c38d4 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:04:07.897884 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf424df-07c6-4a8a-ba23-90feca9c38d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:04:07.930941 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-5bf424df-07c6-4a8a-ba23-90feca9c38d4 None None] MFA Rules not processed for user `27653498cd0a4e089f1cbc43470182ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 15:04:07.932331 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-5bf424df-07c6-4a8a-ba23-90feca9c38d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:04:07.997890 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 15:04:08.014338 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1236/2482] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 15:04:07 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 15:04:08.026547 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca5f9e7-7237-4d57-a44f-afa3d0a61809 service nova] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:04:08.028182 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-eca5f9e7-7237-4d57-a44f-afa3d0a61809 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:04:08.042815 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca5f9e7-7237-4d57-a44f-afa3d0a61809 service nova] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:04:08.049509 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca58a1a-f5ec-4569-9fb8-46b6694d477d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4858bb8621854ec8aba836e7d9050498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:04:08.050128 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca58a1a-f5ec-4569-9fb8-46b6694d477d service nova] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:04:08.050459 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca58a1a-f5ec-4569-9fb8-46b6694d477d service nova] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:04:08.050735 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca58a1a-f5ec-4569-9fb8-46b6694d477d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:04:08.060151 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca58a1a-f5ec-4569-9fb8-46b6694d477d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:04:08.061751 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca58a1a-f5ec-4569-9fb8-46b6694d477d service nova] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:04:08.077102 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1247/2483] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 15:04:08 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 15:04:09.565078 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f514b3a3-a1bd-4bdb-9e5d-7d4c22f144b5 None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:04:09.565505 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f514b3a3-a1bd-4bdb-9e5d-7d4c22f144b5 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:04:09.565638 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f514b3a3-a1bd-4bdb-9e5d-7d4c22f144b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:04:09.586023 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-f514b3a3-a1bd-4bdb-9e5d-7d4c22f144b5 None None] MFA Rules not processed for user `27653498cd0a4e089f1cbc43470182ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 15:04:09.587860 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-f514b3a3-a1bd-4bdb-9e5d-7d4c22f144b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:04:09.595333 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1237/2484] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 15:04:09 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 15:04:10.463971 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca58a1a-f5ec-4569-9fb8-46b6694d477d service nova] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:04:10.465393 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-0ca58a1a-f5ec-4569-9fb8-46b6694d477d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:04:10.483643 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca58a1a-f5ec-4569-9fb8-46b6694d477d service nova] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:04:10.491357 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e399389-a4c9-4915-a73e-22a0ab95d67a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2790149639aa418bbc4e4982e93eda44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:04:10.492176 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e399389-a4c9-4915-a73e-22a0ab95d67a service neutron] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:04:10.493151 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e399389-a4c9-4915-a73e-22a0ab95d67a service neutron] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:04:10.493567 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e399389-a4c9-4915-a73e-22a0ab95d67a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:04:10.495609 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-0e399389-a4c9-4915-a73e-22a0ab95d67a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:04:10.622998 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e399389-a4c9-4915-a73e-22a0ab95d67a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:04:10.624472 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e399389-a4c9-4915-a73e-22a0ab95d67a service neutron] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:04:10.629636 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 15:04:10.637008 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1248/2485] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 15:04:10 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 15:04:12.801400 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f514b3a3-a1bd-4bdb-9e5d-7d4c22f144b5 None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:04:12.828499 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f514b3a3-a1bd-4bdb-9e5d-7d4c22f144b5 None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:04:12.835465 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309e86f8-66eb-4942-8c1f-64c8eb63057e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4858bb8621854ec8aba836e7d9050498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:04:12.836215 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309e86f8-66eb-4942-8c1f-64c8eb63057e service nova] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:04:12.837221 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309e86f8-66eb-4942-8c1f-64c8eb63057e service nova] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:04:12.837705 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309e86f8-66eb-4942-8c1f-64c8eb63057e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:04:12.839625 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-309e86f8-66eb-4942-8c1f-64c8eb63057e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:04:12.852871 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309e86f8-66eb-4942-8c1f-64c8eb63057e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:04:12.854687 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309e86f8-66eb-4942-8c1f-64c8eb63057e service nova] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:04:12.869530 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1238/2486] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 15:04:12 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 15:04:17.730856 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e399389-a4c9-4915-a73e-22a0ab95d67a service neutron] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:04:17.745141 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e399389-a4c9-4915-a73e-22a0ab95d67a service neutron] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:04:17.752500 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9432946-98de-45cc-84c9-ed4b02630282 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4858bb8621854ec8aba836e7d9050498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:04:17.753212 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9432946-98de-45cc-84c9-ed4b02630282 service nova] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:04:17.753493 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9432946-98de-45cc-84c9-ed4b02630282 service nova] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:04:17.753766 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9432946-98de-45cc-84c9-ed4b02630282 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:04:17.755523 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-d9432946-98de-45cc-84c9-ed4b02630282 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:04:17.768689 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9432946-98de-45cc-84c9-ed4b02630282 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:04:17.771264 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9432946-98de-45cc-84c9-ed4b02630282 service nova] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:04:17.783912 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1249/2487] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 15:04:17 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 15:04:18.075168 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b92c1a2-aea7-4189-b1a5-d11877f73095 None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:04:18.075647 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b92c1a2-aea7-4189-b1a5-d11877f73095 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:04:18.075757 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b92c1a2-aea7-4189-b1a5-d11877f73095 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:04:18.095499 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-7b92c1a2-aea7-4189-b1a5-d11877f73095 None None] MFA Rules not processed for user `24b6f275ffb04ffeae3c2097bfd040f5`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 15:04:18.096719 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-7b92c1a2-aea7-4189-b1a5-d11877f73095 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:04:18.151211 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1239/2488] 10.222.0.33 () {58 vars in 1009 bytes} [Thu Sep 1 15:04:18 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 15:04:18.197031 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9432946-98de-45cc-84c9-ed4b02630282 service nova] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:04:18.198509 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-d9432946-98de-45cc-84c9-ed4b02630282 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:04:18.264421 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9432946-98de-45cc-84c9-ed4b02630282 service nova] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:04:18.269440 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3e4482-dd14-4c39-a22e-71aac1262a3a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73a5bda5a6ea46e9863ccafd57dee339', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:04:18.270040 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3e4482-dd14-4c39-a22e-71aac1262a3a service swift] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:04:18.270357 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3e4482-dd14-4c39-a22e-71aac1262a3a service swift] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:04:18.270631 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3e4482-dd14-4c39-a22e-71aac1262a3a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:04:18.279416 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3e4482-dd14-4c39-a22e-71aac1262a3a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:04:18.280988 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3e4482-dd14-4c39-a22e-71aac1262a3a service swift] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:04:18.288796 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1250/2489] 10.222.0.33 () {60 vars in 1384 bytes} [Thu Sep 1 15:04:18 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 01 15:04:28.143864 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=36 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 15:04:28.285602 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=39 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 15:04:31.453022 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c29cdaa-63ab-47cf-8891-6c5a9860ff43 None None] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:04:31.453508 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c29cdaa-63ab-47cf-8891-6c5a9860ff43 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:04:31.453619 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c29cdaa-63ab-47cf-8891-6c5a9860ff43 None None] PATH_INFO: `/` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:04:31.455162 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1240/2490] 10.222.0.33 () {54 vars in 921 bytes} [Thu Sep 1 15:04:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 15:04:31.461581 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a25fbe-f2e0-4124-89d3-ddb4555e3a58 None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:04:31.462456 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a25fbe-f2e0-4124-89d3-ddb4555e3a58 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:04:31.462901 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a25fbe-f2e0-4124-89d3-ddb4555e3a58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:04:31.491573 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-f2a25fbe-f2e0-4124-89d3-ddb4555e3a58 None None] MFA Rules not processed for user `4858bb8621854ec8aba836e7d9050498`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 15:04:31.493331 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-f2a25fbe-f2e0-4124-89d3-ddb4555e3a58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:04:31.502633 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1251/2491] 10.222.0.33 () {58 vars in 1018 bytes} [Thu Sep 1 15:04:31 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 15:04:31.515683 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c29cdaa-63ab-47cf-8891-6c5a9860ff43 None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:04:31.517601 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-1c29cdaa-63ab-47cf-8891-6c5a9860ff43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:04:31.590342 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c29cdaa-63ab-47cf-8891-6c5a9860ff43 None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:04:31.599393 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d04520-d9e7-4539-8ae0-94803af462db service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3069f72ce98049b99c9d98229a62bb51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:04:31.600138 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d04520-d9e7-4539-8ae0-94803af462db service glance] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:04:31.600422 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d04520-d9e7-4539-8ae0-94803af462db service glance] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:04:31.600752 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d04520-d9e7-4539-8ae0-94803af462db service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:04:31.612439 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d04520-d9e7-4539-8ae0-94803af462db service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:04:31.613881 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d04520-d9e7-4539-8ae0-94803af462db service glance] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:04:31.625567 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1241/2492] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 15:04:31 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 15:04:31.696159 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3851b22-f2d4-4d8c-8efd-1bf3af19b01f None None] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:04:31.696436 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3851b22-f2d4-4d8c-8efd-1bf3af19b01f None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:04:31.696704 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3851b22-f2d4-4d8c-8efd-1bf3af19b01f None None] PATH_INFO: `/` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:04:31.698023 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1252/2493] 10.222.0.33 () {54 vars in 921 bytes} [Thu Sep 1 15:04:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 15:04:31.702922 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e2effb-806f-4896-a9c7-7a35feb5137d None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:04:31.703205 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e2effb-806f-4896-a9c7-7a35feb5137d None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:04:31.703470 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e2effb-806f-4896-a9c7-7a35feb5137d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:04:31.722676 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-16e2effb-806f-4896-a9c7-7a35feb5137d None None] MFA Rules not processed for user `2790149639aa418bbc4e4982e93eda44`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 15:04:31.723868 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-16e2effb-806f-4896-a9c7-7a35feb5137d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:04:31.730755 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1242/2494] 10.222.0.33 () {58 vars in 1018 bytes} [Thu Sep 1 15:04:31 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 15:04:31.742762 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3851b22-f2d4-4d8c-8efd-1bf3af19b01f None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:04:31.753503 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3851b22-f2d4-4d8c-8efd-1bf3af19b01f None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:04:31.756979 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7e86e7-ae3d-4d47-8930-5859693768db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2790149639aa418bbc4e4982e93eda44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:04:31.757583 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7e86e7-ae3d-4d47-8930-5859693768db service neutron] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:04:31.757858 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7e86e7-ae3d-4d47-8930-5859693768db service neutron] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:04:31.758125 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7e86e7-ae3d-4d47-8930-5859693768db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:04:31.765090 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7e86e7-ae3d-4d47-8930-5859693768db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:04:31.766538 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7e86e7-ae3d-4d47-8930-5859693768db service neutron] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:04:31.774350 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1253/2495] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 15:04:31 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 33 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 15:04:31.857483 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafd7c1f-dfd3-47a5-8a2b-9027d41cd65a None None] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:04:31.857763 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafd7c1f-dfd3-47a5-8a2b-9027d41cd65a None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:04:31.858030 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafd7c1f-dfd3-47a5-8a2b-9027d41cd65a None None] PATH_INFO: `/` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:04:31.859405 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1243/2496] 10.222.0.33 () {54 vars in 921 bytes} [Thu Sep 1 15:04:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 15:04:31.864839 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5302c180-ae8d-4438-8452-fad413755681 None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:04:31.865118 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5302c180-ae8d-4438-8452-fad413755681 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:04:31.865383 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5302c180-ae8d-4438-8452-fad413755681 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:04:31.885194 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-5302c180-ae8d-4438-8452-fad413755681 None None] MFA Rules not processed for user `4858bb8621854ec8aba836e7d9050498`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 15:04:31.886418 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-5302c180-ae8d-4438-8452-fad413755681 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:04:31.895231 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1254/2497] 10.222.0.33 () {58 vars in 1018 bytes} [Thu Sep 1 15:04:31 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 15:04:31.905846 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafd7c1f-dfd3-47a5-8a2b-9027d41cd65a None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:04:31.917524 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafd7c1f-dfd3-47a5-8a2b-9027d41cd65a None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:04:31.921003 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87806e23-df8c-46a1-bc59-c3f0e727568f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2790149639aa418bbc4e4982e93eda44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:04:31.921584 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87806e23-df8c-46a1-bc59-c3f0e727568f service neutron] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:04:31.922109 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87806e23-df8c-46a1-bc59-c3f0e727568f service neutron] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:04:31.922434 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87806e23-df8c-46a1-bc59-c3f0e727568f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:04:31.932300 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87806e23-df8c-46a1-bc59-c3f0e727568f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:04:31.933720 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87806e23-df8c-46a1-bc59-c3f0e727568f service neutron] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:04:31.944191 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1244/2498] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 15:04:31 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 15:04:33.753789 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0060f72a-ca70-4ac1-a862-d9553ff7e3f0 None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:04:33.754282 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0060f72a-ca70-4ac1-a862-d9553ff7e3f0 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:04:33.754379 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0060f72a-ca70-4ac1-a862-d9553ff7e3f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:04:33.773487 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-0060f72a-ca70-4ac1-a862-d9553ff7e3f0 None None] MFA Rules not processed for user `24b6f275ffb04ffeae3c2097bfd040f5`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 15:04:33.774724 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-0060f72a-ca70-4ac1-a862-d9553ff7e3f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:04:33.781783 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1255/2499] 10.222.0.33 () {58 vars in 1009 bytes} [Thu Sep 1 15:04:33 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 15:04:33.794050 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87806e23-df8c-46a1-bc59-c3f0e727568f service neutron] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:04:33.805587 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87806e23-df8c-46a1-bc59-c3f0e727568f service neutron] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:04:33.810692 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b254d6ff-8094-4744-b8d7-8c321d81ad94 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73a5bda5a6ea46e9863ccafd57dee339', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:04:33.811291 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b254d6ff-8094-4744-b8d7-8c321d81ad94 service swift] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:04:33.811570 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b254d6ff-8094-4744-b8d7-8c321d81ad94 service swift] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:04:33.811869 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b254d6ff-8094-4744-b8d7-8c321d81ad94 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:04:33.820687 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b254d6ff-8094-4744-b8d7-8c321d81ad94 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:04:33.822376 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b254d6ff-8094-4744-b8d7-8c321d81ad94 service swift] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:04:33.832389 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1245/2500] 10.222.0.33 () {60 vars in 1384 bytes} [Thu Sep 1 15:04:33 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 01 15:04:43.768278 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=15 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 15:04:43.827857 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=41 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 15:04:49.683253 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b5a0b0-3c10-4bb7-983c-d009a81eb95f None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:04:49.683653 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b5a0b0-3c10-4bb7-983c-d009a81eb95f None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:04:49.683790 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b5a0b0-3c10-4bb7-983c-d009a81eb95f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:04:49.702304 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-95b5a0b0-3c10-4bb7-983c-d009a81eb95f None None] MFA Rules not processed for user `24b6f275ffb04ffeae3c2097bfd040f5`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 15:04:49.703505 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-95b5a0b0-3c10-4bb7-983c-d009a81eb95f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:04:49.710813 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1256/2501] 10.222.0.33 () {58 vars in 967 bytes} [Thu Sep 1 15:04:49 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 15:04:49.730424 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b254d6ff-8094-4744-b8d7-8c321d81ad94 service swift] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:04:49.741881 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b254d6ff-8094-4744-b8d7-8c321d81ad94 service swift] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:04:49.747083 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068344b2-aa2e-4e4f-a944-45637084863f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73a5bda5a6ea46e9863ccafd57dee339', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:04:49.747756 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068344b2-aa2e-4e4f-a944-45637084863f service swift] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:04:49.748058 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068344b2-aa2e-4e4f-a944-45637084863f service swift] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:04:49.748325 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068344b2-aa2e-4e4f-a944-45637084863f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:04:49.757073 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068344b2-aa2e-4e4f-a944-45637084863f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:04:49.758334 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068344b2-aa2e-4e4f-a944-45637084863f service swift] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:04:49.767027 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1246/2502] 10.222.0.33 () {60 vars in 1384 bytes} [Thu Sep 1 15:04:49 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 01 15:04:59.697462 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=3 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 15:04:59.763964 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=6 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 15:05:15.705614 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14f672a-749b-4706-a00c-e77fffa8efbd None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:05:15.706165 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14f672a-749b-4706-a00c-e77fffa8efbd None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:05:15.706293 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14f672a-749b-4706-a00c-e77fffa8efbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:05:15.725574 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-b14f672a-749b-4706-a00c-e77fffa8efbd None None] MFA Rules not processed for user `27653498cd0a4e089f1cbc43470182ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 15:05:15.726799 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-b14f672a-749b-4706-a00c-e77fffa8efbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:05:15.734136 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1257/2503] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 15:05:15 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 15:05:24.144555 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068344b2-aa2e-4e4f-a944-45637084863f service swift] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:05:24.154028 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068344b2-aa2e-4e4f-a944-45637084863f service swift] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:05:24.158381 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e60a4a0-92af-4d15-8372-f377c934f79f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2790149639aa418bbc4e4982e93eda44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:05:24.159082 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e60a4a0-92af-4d15-8372-f377c934f79f service neutron] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:05:24.159362 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e60a4a0-92af-4d15-8372-f377c934f79f service neutron] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:05:24.159658 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e60a4a0-92af-4d15-8372-f377c934f79f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:05:24.161395 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-8e60a4a0-92af-4d15-8372-f377c934f79f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:05:24.173464 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e60a4a0-92af-4d15-8372-f377c934f79f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:05:24.174937 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e60a4a0-92af-4d15-8372-f377c934f79f service neutron] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:05:24.185245 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1247/2504] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 15:05:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 15:05:25.720966 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=3 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 15:05:27.952023 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14f672a-749b-4706-a00c-e77fffa8efbd None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:05:27.953474 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-b14f672a-749b-4706-a00c-e77fffa8efbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:05:28.053819 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e60a4a0-92af-4d15-8372-f377c934f79f service neutron] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:05:28.056130 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-8e60a4a0-92af-4d15-8372-f377c934f79f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:05:28.074174 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e60a4a0-92af-4d15-8372-f377c934f79f service neutron] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:05:28.082046 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad8c256-2547-4c51-9a24-b2f10307ba6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:05:28.082866 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad8c256-2547-4c51-9a24-b2f10307ba6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:05:28.083009 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad8c256-2547-4c51-9a24-b2f10307ba6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:05:28.083478 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad8c256-2547-4c51-9a24-b2f10307ba6b admin admin] PATH_INFO: `/v3/users/82255a5060674d77a554e34f89ab1bff` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:05:28.083610 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14f672a-749b-4706-a00c-e77fffa8efbd None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:05:28.084803 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad8c256-2547-4c51-9a24-b2f10307ba6b admin admin] RBAC: Authorizing `identity:delete_user(user_id=82255a5060674d77a554e34f89ab1bff)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:05:28.086802 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad8c256-2547-4c51-9a24-b2f10307ba6b admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:05:28.089001 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064cdef7-3b2a-49dd-a256-0a60b2e4ce9b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f312c8dcabe48ef9ea453364ade38ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:05:28.089717 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064cdef7-3b2a-49dd-a256-0a60b2e4ce9b service placement] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:05:28.090429 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064cdef7-3b2a-49dd-a256-0a60b2e4ce9b service placement] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:05:28.090723 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064cdef7-3b2a-49dd-a256-0a60b2e4ce9b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:05:28.092807 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-064cdef7-3b2a-49dd-a256-0a60b2e4ce9b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:05:28.116277 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064cdef7-3b2a-49dd-a256-0a60b2e4ce9b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:05:28.118054 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064cdef7-3b2a-49dd-a256-0a60b2e4ce9b service placement] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:05:28.132715 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1258/2505] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 15:05:27 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 15:05:28.181903 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-9ad8c256-2547-4c51-9a24-b2f10307ba6b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '82255a5060674d77a554e34f89ab1bff'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:05:28.199054 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-9ad8c256-2547-4c51-9a24-b2f10307ba6b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '82255a5060674d77a554e34f89ab1bff'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:05:28.199734 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-9ad8c256-2547-4c51-9a24-b2f10307ba6b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '82255a5060674d77a554e34f89ab1bff'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:05:28.212303 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-9ad8c256-2547-4c51-9a24-b2f10307ba6b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '82255a5060674d77a554e34f89ab1bff'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:05:28.212968 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-9ad8c256-2547-4c51-9a24-b2f10307ba6b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '82255a5060674d77a554e34f89ab1bff'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:05:28.225633 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1248/2506] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Sep 1 15:05:28 2022] DELETE /identity/v3/users/82255a5060674d77a554e34f89ab1bff => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 15:05:28.505043 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064cdef7-3b2a-49dd-a256-0a60b2e4ce9b service placement] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:05:28.519683 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064cdef7-3b2a-49dd-a256-0a60b2e4ce9b service placement] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:05:28.586729 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab6ceee-ecb1-4d05-9657-e407be8566af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:05:28.587367 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab6ceee-ecb1-4d05-9657-e407be8566af admin admin] REQUEST_METHOD: `DELETE` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:05:28.587642 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab6ceee-ecb1-4d05-9657-e407be8566af admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:05:28.587907 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab6ceee-ecb1-4d05-9657-e407be8566af admin admin] PATH_INFO: `/v3/projects/b0202aa985634c8b8b241ce016600b19` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:05:28.589049 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab6ceee-ecb1-4d05-9657-e407be8566af admin admin] RBAC: Authorizing `identity:delete_project(project_id=b0202aa985634c8b8b241ce016600b19)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:05:28.591542 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab6ceee-ecb1-4d05-9657-e407be8566af admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:05:28.644072 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-3ab6ceee-ecb1-4d05-9657-e407be8566af admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b0202aa985634c8b8b241ce016600b19', 'request_id': 'req-3ab6ceee-ecb1-4d05-9657-e407be8566af'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:05:28.650203 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-3ab6ceee-ecb1-4d05-9657-e407be8566af admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b0202aa985634c8b8b241ce016600b19', 'request_id': 'req-3ab6ceee-ecb1-4d05-9657-e407be8566af'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:05:28.662679 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-3ab6ceee-ecb1-4d05-9657-e407be8566af admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b0202aa985634c8b8b241ce016600b19', 'request_id': 'req-3ab6ceee-ecb1-4d05-9657-e407be8566af'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:05:28.677885 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-3ab6ceee-ecb1-4d05-9657-e407be8566af admin admin] The token cache is being invalidate because project b0202aa985634c8b8b241ce016600b19 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81342) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 15:05:28.678181 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-3ab6ceee-ecb1-4d05-9657-e407be8566af admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:05:28.678879 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-3ab6ceee-ecb1-4d05-9657-e407be8566af admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:05:28.680575 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1259/2507] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Sep 1 15:05:28 2022] DELETE /identity/v3/projects/b0202aa985634c8b8b241ce016600b19 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 15:05:28.686080 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6d4000-2c34-4f55-945d-0b2fe6f664c9 None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:05:28.686498 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6d4000-2c34-4f55-945d-0b2fe6f664c9 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:05:28.686851 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6d4000-2c34-4f55-945d-0b2fe6f664c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:05:28.707326 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-7f6d4000-2c34-4f55-945d-0b2fe6f664c9 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 15:05:28.708557 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-7f6d4000-2c34-4f55-945d-0b2fe6f664c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:05:28.769984 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1249/2508] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 15:05:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 15:05:28.778203 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd341927-1117-493b-9fad-c7e840934ef7 None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:05:28.778522 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd341927-1117-493b-9fad-c7e840934ef7 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:05:28.778800 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd341927-1117-493b-9fad-c7e840934ef7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:05:28.814750 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-cd341927-1117-493b-9fad-c7e840934ef7 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 15:05:28.815598 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-cd341927-1117-493b-9fad-c7e840934ef7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:05:28.824407 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1260/2509] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 15:05:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 15:05:28.828923 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6d4000-2c34-4f55-945d-0b2fe6f664c9 None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:05:28.841575 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6d4000-2c34-4f55-945d-0b2fe6f664c9 None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:05:28.847396 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9231240-d5f0-44d9-83ee-2915475ff9d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:05:28.848132 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9231240-d5f0-44d9-83ee-2915475ff9d0 admin admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:05:28.849137 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9231240-d5f0-44d9-83ee-2915475ff9d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:05:28.849504 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9231240-d5f0-44d9-83ee-2915475ff9d0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:05:28.850228 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9231240-d5f0-44d9-83ee-2915475ff9d0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81341) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 15:05:28.850626 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9231240-d5f0-44d9-83ee-2915475ff9d0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:05:28.852509 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9231240-d5f0-44d9-83ee-2915475ff9d0 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:05:28.867024 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1250/2510] 10.222.0.33 () {58 vars in 1153 bytes} [Thu Sep 1 15:05:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 15:05:28.872497 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd341927-1117-493b-9fad-c7e840934ef7 None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:05:28.884044 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd341927-1117-493b-9fad-c7e840934ef7 None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:05:28.889322 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9835f795-2945-46dd-8da2-902f39c866aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:05:28.890111 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9835f795-2945-46dd-8da2-902f39c866aa admin admin] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:05:28.890583 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9835f795-2945-46dd-8da2-902f39c866aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:05:28.890853 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9835f795-2945-46dd-8da2-902f39c866aa admin admin] PATH_INFO: `/v3/projects` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:05:28.891488 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9835f795-2945-46dd-8da2-902f39c866aa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:05:28.894138 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9835f795-2945-46dd-8da2-902f39c866aa admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:05:28.906728 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 15:05:28.934397 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1261/2511] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Sep 1 15:05:28 2022] POST /identity/v3/projects => generated 373 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 15:05:28.935983 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9231240-d5f0-44d9-83ee-2915475ff9d0 admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:05:28.946750 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9231240-d5f0-44d9-83ee-2915475ff9d0 admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:05:28.991643 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 15:05:29.008753 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af810e5-7b70-415e-bb8b-1f8adc32e7f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:05:29.009361 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af810e5-7b70-415e-bb8b-1f8adc32e7f3 admin admin] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:05:29.009645 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af810e5-7b70-415e-bb8b-1f8adc32e7f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:05:29.009912 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af810e5-7b70-415e-bb8b-1f8adc32e7f3 admin admin] PATH_INFO: `/v3/users` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:05:29.010601 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af810e5-7b70-415e-bb8b-1f8adc32e7f3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:05:29.012634 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af810e5-7b70-415e-bb8b-1f8adc32e7f3 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:05:29.039494 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1251/2512] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Sep 1 15:05:28 2022] POST /identity/v3/users => generated 287 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 15:05:29.044390 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9835f795-2945-46dd-8da2-902f39c866aa admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:05:29.055877 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9835f795-2945-46dd-8da2-902f39c866aa admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:05:29.061608 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f6d88a-8350-47c1-9c06-a236603b9631 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:05:29.062222 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f6d88a-8350-47c1-9c06-a236603b9631 admin admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:05:29.062534 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f6d88a-8350-47c1-9c06-a236603b9631 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:05:29.062804 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f6d88a-8350-47c1-9c06-a236603b9631 admin admin] PATH_INFO: `/v3/roles` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:05:29.063512 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f6d88a-8350-47c1-9c06-a236603b9631 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:05:29.065107 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f6d88a-8350-47c1-9c06-a236603b9631 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:05:29.076105 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1262/2513] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 1 15:05:29 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 15:05:29.080494 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af810e5-7b70-415e-bb8b-1f8adc32e7f3 admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:05:29.092619 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af810e5-7b70-415e-bb8b-1f8adc32e7f3 admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:05:29.101392 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddccd2a-b316-4fb1-82ab-89ba937dfcbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:05:29.102117 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddccd2a-b316-4fb1-82ab-89ba937dfcbb admin admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:05:29.102415 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddccd2a-b316-4fb1-82ab-89ba937dfcbb admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:05:29.102787 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddccd2a-b316-4fb1-82ab-89ba937dfcbb admin admin] PATH_INFO: `/v3/roles` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:05:29.103443 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddccd2a-b316-4fb1-82ab-89ba937dfcbb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:05:29.106127 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddccd2a-b316-4fb1-82ab-89ba937dfcbb admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:05:29.119215 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1252/2514] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 1 15:05:29 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 15:05:29.124409 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f6d88a-8350-47c1-9c06-a236603b9631 admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:05:29.135289 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f6d88a-8350-47c1-9c06-a236603b9631 admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:05:29.140498 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dce5f2-5bc2-4234-94e0-e47449946d88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:05:29.141114 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dce5f2-5bc2-4234-94e0-e47449946d88 admin admin] REQUEST_METHOD: `PUT` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:05:29.141391 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dce5f2-5bc2-4234-94e0-e47449946d88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:05:29.141660 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dce5f2-5bc2-4234-94e0-e47449946d88 admin admin] PATH_INFO: `/v3/projects/20daf23edeb447dfab9bf828170bd858/users/d4522eb2588b4ea68495107f261d3427/roles/5d1feef0e13d4874928bd86258c4e3b2` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:05:29.158525 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37dce5f2-5bc2-4234-94e0-e47449946d88 admin admin] RBAC: Authorizing `identity:create_grant(project_id=20daf23edeb447dfab9bf828170bd858, user_id=d4522eb2588b4ea68495107f261d3427, role_id=5d1feef0e13d4874928bd86258c4e3b2)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:05:29.162013 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37dce5f2-5bc2-4234-94e0-e47449946d88 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:05:29.176199 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1263/2515] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 1 15:05:29 2022] PUT /identity/v3/projects/20daf23edeb447dfab9bf828170bd858/users/d4522eb2588b4ea68495107f261d3427/roles/5d1feef0e13d4874928bd86258c4e3b2 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 15:05:29.188438 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a4a15d-b134-4b98-a1d2-d07dc1984cb3 None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:05:29.188730 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a4a15d-b134-4b98-a1d2-d07dc1984cb3 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:05:29.188997 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a4a15d-b134-4b98-a1d2-d07dc1984cb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:05:29.218714 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-27a4a15d-b134-4b98-a1d2-d07dc1984cb3 None None] MFA Rules not processed for user `d4522eb2588b4ea68495107f261d3427`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 15:05:29.220043 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-27a4a15d-b134-4b98-a1d2-d07dc1984cb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:05:29.287319 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1253/2516] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 15:05:29 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 15:05:29.299194 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dce5f2-5bc2-4234-94e0-e47449946d88 admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:05:29.301804 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-37dce5f2-5bc2-4234-94e0-e47449946d88 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:05:29.381688 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dce5f2-5bc2-4234-94e0-e47449946d88 admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:05:29.387691 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fcf0df-2dc1-482a-8592-07606768654a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4858bb8621854ec8aba836e7d9050498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:05:29.388293 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fcf0df-2dc1-482a-8592-07606768654a service nova] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:05:29.388569 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fcf0df-2dc1-482a-8592-07606768654a service nova] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:05:29.388838 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fcf0df-2dc1-482a-8592-07606768654a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:05:29.396646 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fcf0df-2dc1-482a-8592-07606768654a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:05:29.398220 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fcf0df-2dc1-482a-8592-07606768654a service nova] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:05:29.407623 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1264/2517] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 15:05:29 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 15:05:31.289252 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a4a15d-b134-4b98-a1d2-d07dc1984cb3 None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:05:31.290368 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-27a4a15d-b134-4b98-a1d2-d07dc1984cb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:05:31.359304 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a4a15d-b134-4b98-a1d2-d07dc1984cb3 None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:05:31.362809 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105152e0-d8ce-4c0d-a922-fdf1e5384c46 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2790149639aa418bbc4e4982e93eda44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:05:31.363406 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105152e0-d8ce-4c0d-a922-fdf1e5384c46 service neutron] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:05:31.363683 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105152e0-d8ce-4c0d-a922-fdf1e5384c46 service neutron] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:05:31.363953 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105152e0-d8ce-4c0d-a922-fdf1e5384c46 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:05:31.365682 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-105152e0-d8ce-4c0d-a922-fdf1e5384c46 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:05:31.427137 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105152e0-d8ce-4c0d-a922-fdf1e5384c46 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:05:31.428629 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105152e0-d8ce-4c0d-a922-fdf1e5384c46 service neutron] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:05:31.438982 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1254/2518] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 15:05:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 15:05:32.141120 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fcf0df-2dc1-482a-8592-07606768654a service nova] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:05:32.150982 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fcf0df-2dc1-482a-8592-07606768654a service nova] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:05:32.156444 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f5a398-4c79-4b10-b8bf-80e8bac1aab6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:05:32.157577 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f5a398-4c79-4b10-b8bf-80e8bac1aab6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:05:32.157577 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f5a398-4c79-4b10-b8bf-80e8bac1aab6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:05:32.157856 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f5a398-4c79-4b10-b8bf-80e8bac1aab6 admin admin] PATH_INFO: `/v3/users/d4522eb2588b4ea68495107f261d3427` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:05:32.159020 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f5a398-4c79-4b10-b8bf-80e8bac1aab6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d4522eb2588b4ea68495107f261d3427)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:05:32.160937 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f5a398-4c79-4b10-b8bf-80e8bac1aab6 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:05:32.212114 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-e5f5a398-4c79-4b10-b8bf-80e8bac1aab6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd4522eb2588b4ea68495107f261d3427'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:05:32.223240 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-e5f5a398-4c79-4b10-b8bf-80e8bac1aab6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd4522eb2588b4ea68495107f261d3427'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:05:32.224096 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-e5f5a398-4c79-4b10-b8bf-80e8bac1aab6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd4522eb2588b4ea68495107f261d3427'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:05:32.245886 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-e5f5a398-4c79-4b10-b8bf-80e8bac1aab6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd4522eb2588b4ea68495107f261d3427'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:05:32.246631 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-e5f5a398-4c79-4b10-b8bf-80e8bac1aab6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd4522eb2588b4ea68495107f261d3427'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:05:32.259808 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1265/2519] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Sep 1 15:05:32 2022] DELETE /identity/v3/users/d4522eb2588b4ea68495107f261d3427 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 15:05:32.265328 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2800ae9f-6ba7-4ac9-ae3f-2856f94968ba None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:05:32.265670 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2800ae9f-6ba7-4ac9-ae3f-2856f94968ba None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:05:32.265950 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2800ae9f-6ba7-4ac9-ae3f-2856f94968ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:05:32.274773 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 15:05:32.288648 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-2800ae9f-6ba7-4ac9-ae3f-2856f94968ba None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 15:05:32.289964 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-2800ae9f-6ba7-4ac9-ae3f-2856f94968ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:05:32.367928 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1255/2520] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 15:05:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 15:05:32.381162 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f5a398-4c79-4b10-b8bf-80e8bac1aab6 admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:05:32.382888 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-e5f5a398-4c79-4b10-b8bf-80e8bac1aab6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:05:32.399866 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 15:05:32.457851 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f5a398-4c79-4b10-b8bf-80e8bac1aab6 admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:05:32.462015 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870df3d6-083d-4726-8adc-451b413428dd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2790149639aa418bbc4e4982e93eda44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:05:32.462812 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870df3d6-083d-4726-8adc-451b413428dd service neutron] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:05:32.463042 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870df3d6-083d-4726-8adc-451b413428dd service neutron] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:05:32.463314 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870df3d6-083d-4726-8adc-451b413428dd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:05:32.474328 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870df3d6-083d-4726-8adc-451b413428dd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:05:32.475839 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870df3d6-083d-4726-8adc-451b413428dd service neutron] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:05:32.487268 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1266/2521] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 15:05:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 15:05:32.772736 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2800ae9f-6ba7-4ac9-ae3f-2856f94968ba None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:05:32.774356 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-2800ae9f-6ba7-4ac9-ae3f-2856f94968ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:05:32.787182 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2800ae9f-6ba7-4ac9-ae3f-2856f94968ba None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:05:32.802346 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0302cc1-123e-47f3-b822-6e54f2d65774 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:05:32.803321 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0302cc1-123e-47f3-b822-6e54f2d65774 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:05:32.803667 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0302cc1-123e-47f3-b822-6e54f2d65774 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:05:32.804021 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0302cc1-123e-47f3-b822-6e54f2d65774 admin admin] PATH_INFO: `/v3/projects/20daf23edeb447dfab9bf828170bd858` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:05:32.805413 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0302cc1-123e-47f3-b822-6e54f2d65774 admin admin] RBAC: Authorizing `identity:delete_project(project_id=20daf23edeb447dfab9bf828170bd858)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:05:32.807755 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0302cc1-123e-47f3-b822-6e54f2d65774 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:05:32.862719 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-b0302cc1-123e-47f3-b822-6e54f2d65774 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '20daf23edeb447dfab9bf828170bd858', 'request_id': 'req-b0302cc1-123e-47f3-b822-6e54f2d65774'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:05:32.869009 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-b0302cc1-123e-47f3-b822-6e54f2d65774 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '20daf23edeb447dfab9bf828170bd858', 'request_id': 'req-b0302cc1-123e-47f3-b822-6e54f2d65774'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:05:32.877677 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-b0302cc1-123e-47f3-b822-6e54f2d65774 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '20daf23edeb447dfab9bf828170bd858', 'request_id': 'req-b0302cc1-123e-47f3-b822-6e54f2d65774'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:05:32.888941 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-b0302cc1-123e-47f3-b822-6e54f2d65774 admin admin] The token cache is being invalidate because project 20daf23edeb447dfab9bf828170bd858 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81341) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 15:05:32.889226 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-b0302cc1-123e-47f3-b822-6e54f2d65774 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:05:32.889866 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-b0302cc1-123e-47f3-b822-6e54f2d65774 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:05:32.891429 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1256/2522] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Sep 1 15:05:32 2022] DELETE /identity/v3/projects/20daf23edeb447dfab9bf828170bd858 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 15:05:32.896336 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10679baa-948c-4fbf-8ca5-f18318efb02a None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:05:32.896688 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10679baa-948c-4fbf-8ca5-f18318efb02a None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:05:32.897046 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10679baa-948c-4fbf-8ca5-f18318efb02a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:05:32.917669 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-10679baa-948c-4fbf-8ca5-f18318efb02a None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 15:05:32.918926 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-10679baa-948c-4fbf-8ca5-f18318efb02a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:05:32.981308 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1267/2523] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 15:05:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 15:05:32.988740 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cba225b-e2f6-413a-81c2-b565527f6b04 None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:05:32.989093 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cba225b-e2f6-413a-81c2-b565527f6b04 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:05:32.989461 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cba225b-e2f6-413a-81c2-b565527f6b04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:05:33.010792 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-1cba225b-e2f6-413a-81c2-b565527f6b04 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 15:05:33.012000 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-1cba225b-e2f6-413a-81c2-b565527f6b04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:05:33.020123 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1257/2524] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 15:05:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 15:05:33.024422 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10679baa-948c-4fbf-8ca5-f18318efb02a None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:05:33.034748 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10679baa-948c-4fbf-8ca5-f18318efb02a None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:05:33.040983 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589ac83b-cb87-4d43-a604-8d57ad03a8e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:05:33.041466 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589ac83b-cb87-4d43-a604-8d57ad03a8e8 admin admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:05:33.041813 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589ac83b-cb87-4d43-a604-8d57ad03a8e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:05:33.042089 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589ac83b-cb87-4d43-a604-8d57ad03a8e8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:05:33.044484 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589ac83b-cb87-4d43-a604-8d57ad03a8e8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 15:05:33.044863 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589ac83b-cb87-4d43-a604-8d57ad03a8e8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:05:33.046784 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589ac83b-cb87-4d43-a604-8d57ad03a8e8 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:05:33.062854 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1268/2525] 10.222.0.33 () {58 vars in 1153 bytes} [Thu Sep 1 15:05:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 15:05:33.063702 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cba225b-e2f6-413a-81c2-b565527f6b04 None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:05:33.073765 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cba225b-e2f6-413a-81c2-b565527f6b04 None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:05:33.078562 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f8f558-3096-4dde-8a37-54755f9ac578 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:05:33.079158 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f8f558-3096-4dde-8a37-54755f9ac578 admin admin] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:05:33.079434 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f8f558-3096-4dde-8a37-54755f9ac578 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:05:33.079704 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f8f558-3096-4dde-8a37-54755f9ac578 admin admin] PATH_INFO: `/v3/projects` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:05:33.080347 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f8f558-3096-4dde-8a37-54755f9ac578 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:05:33.082290 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f8f558-3096-4dde-8a37-54755f9ac578 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:05:33.118007 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1258/2526] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Sep 1 15:05:33 2022] POST /identity/v3/projects => generated 381 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 15:05:33.119447 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589ac83b-cb87-4d43-a604-8d57ad03a8e8 admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:05:33.130529 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589ac83b-cb87-4d43-a604-8d57ad03a8e8 admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:05:33.197285 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b071954-04ee-40f3-acc2-7356119e0b39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:05:33.198149 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b071954-04ee-40f3-acc2-7356119e0b39 admin admin] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:05:33.198586 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b071954-04ee-40f3-acc2-7356119e0b39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:05:33.198916 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b071954-04ee-40f3-acc2-7356119e0b39 admin admin] PATH_INFO: `/v3/users` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:05:33.199742 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b071954-04ee-40f3-acc2-7356119e0b39 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:05:33.202035 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b071954-04ee-40f3-acc2-7356119e0b39 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:05:33.226043 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1269/2527] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Sep 1 15:05:33 2022] POST /identity/v3/users => generated 291 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 15:05:33.230141 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f8f558-3096-4dde-8a37-54755f9ac578 admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:05:33.241227 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f8f558-3096-4dde-8a37-54755f9ac578 admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:05:33.248009 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30791d7e-eb14-4fb1-ad2f-58c11e43b7a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:05:33.248166 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30791d7e-eb14-4fb1-ad2f-58c11e43b7a1 admin admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:05:33.248371 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30791d7e-eb14-4fb1-ad2f-58c11e43b7a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:05:33.248637 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30791d7e-eb14-4fb1-ad2f-58c11e43b7a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:05:33.249674 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30791d7e-eb14-4fb1-ad2f-58c11e43b7a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:05:33.251100 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30791d7e-eb14-4fb1-ad2f-58c11e43b7a1 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:05:33.262005 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1259/2528] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 1 15:05:33 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 15:05:33.265895 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b071954-04ee-40f3-acc2-7356119e0b39 admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:05:33.277340 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b071954-04ee-40f3-acc2-7356119e0b39 admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:05:33.281706 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405f2744-2889-4890-85bb-f81075cddb0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:05:33.282411 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405f2744-2889-4890-85bb-f81075cddb0c admin admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:05:33.282777 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405f2744-2889-4890-85bb-f81075cddb0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:05:33.283121 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405f2744-2889-4890-85bb-f81075cddb0c admin admin] PATH_INFO: `/v3/roles` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:05:33.283850 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405f2744-2889-4890-85bb-f81075cddb0c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:05:33.285546 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405f2744-2889-4890-85bb-f81075cddb0c admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:05:33.297349 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1270/2529] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 1 15:05:33 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 15:05:33.301483 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30791d7e-eb14-4fb1-ad2f-58c11e43b7a1 admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:05:33.313250 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30791d7e-eb14-4fb1-ad2f-58c11e43b7a1 admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:05:33.319095 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791d149b-ddf4-4db1-a3f5-bc5511e0c398 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:05:33.319752 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791d149b-ddf4-4db1-a3f5-bc5511e0c398 admin admin] REQUEST_METHOD: `PUT` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:05:33.320027 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791d149b-ddf4-4db1-a3f5-bc5511e0c398 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:05:33.320297 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791d149b-ddf4-4db1-a3f5-bc5511e0c398 admin admin] PATH_INFO: `/v3/projects/05df60fe39dc4be69ef6a912a44f919d/users/cd922700320343beaffdfeeeb97d7122/roles/5d1feef0e13d4874928bd86258c4e3b2` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:05:33.344893 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791d149b-ddf4-4db1-a3f5-bc5511e0c398 admin admin] RBAC: Authorizing `identity:create_grant(project_id=05df60fe39dc4be69ef6a912a44f919d, user_id=cd922700320343beaffdfeeeb97d7122, role_id=5d1feef0e13d4874928bd86258c4e3b2)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:05:33.348417 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791d149b-ddf4-4db1-a3f5-bc5511e0c398 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:05:33.364857 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1260/2530] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 1 15:05:33 2022] PUT /identity/v3/projects/05df60fe39dc4be69ef6a912a44f919d/users/cd922700320343beaffdfeeeb97d7122/roles/5d1feef0e13d4874928bd86258c4e3b2 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 15:05:33.376823 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8ca0d9-c150-46c4-b523-962ac958a1e0 None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:05:33.376823 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8ca0d9-c150-46c4-b523-962ac958a1e0 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:05:33.377184 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8ca0d9-c150-46c4-b523-962ac958a1e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:05:33.408678 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-3c8ca0d9-c150-46c4-b523-962ac958a1e0 None None] MFA Rules not processed for user `cd922700320343beaffdfeeeb97d7122`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 15:05:33.409921 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-3c8ca0d9-c150-46c4-b523-962ac958a1e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:05:33.419942 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 15:05:33.481753 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1271/2531] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 15:05:33 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 15:05:33.486134 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791d149b-ddf4-4db1-a3f5-bc5511e0c398 admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:05:33.514869 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 15:05:33.565690 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791d149b-ddf4-4db1-a3f5-bc5511e0c398 admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:05:33.572809 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9693749-eb46-43ba-862a-72dabdaab5fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:05:33.573797 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9693749-eb46-43ba-862a-72dabdaab5fa admin admin] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:05:33.574306 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9693749-eb46-43ba-862a-72dabdaab5fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:05:33.574712 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9693749-eb46-43ba-862a-72dabdaab5fa admin admin] PATH_INFO: `/v3/projects` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:05:33.575382 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9693749-eb46-43ba-862a-72dabdaab5fa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:05:33.578466 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9693749-eb46-43ba-862a-72dabdaab5fa admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:05:33.607412 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1261/2532] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Sep 1 15:05:33 2022] POST /identity/v3/projects => generated 381 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 15:05:33.611626 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8ca0d9-c150-46c4-b523-962ac958a1e0 None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:05:33.623729 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8ca0d9-c150-46c4-b523-962ac958a1e0 None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:05:33.703414 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd8a32d-83a1-4fa2-8bbf-8ac9c17570df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:05:33.704501 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd8a32d-83a1-4fa2-8bbf-8ac9c17570df admin admin] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:05:33.704501 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd8a32d-83a1-4fa2-8bbf-8ac9c17570df admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:05:33.704706 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd8a32d-83a1-4fa2-8bbf-8ac9c17570df admin admin] PATH_INFO: `/v3/users` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:05:33.705229 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd8a32d-83a1-4fa2-8bbf-8ac9c17570df admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:05:33.707194 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd8a32d-83a1-4fa2-8bbf-8ac9c17570df admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:05:33.728474 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1272/2533] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Sep 1 15:05:33 2022] POST /identity/v3/users => generated 297 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 15:05:33.733040 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9693749-eb46-43ba-862a-72dabdaab5fa admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:05:33.744059 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9693749-eb46-43ba-862a-72dabdaab5fa admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:05:33.749334 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c1ca5a-3321-4a8c-bae8-cb425f6b683a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:05:33.749934 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c1ca5a-3321-4a8c-bae8-cb425f6b683a admin admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:05:33.750320 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c1ca5a-3321-4a8c-bae8-cb425f6b683a admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:05:33.750632 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c1ca5a-3321-4a8c-bae8-cb425f6b683a admin admin] PATH_INFO: `/v3/roles` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:05:33.751270 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c1ca5a-3321-4a8c-bae8-cb425f6b683a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:05:33.752883 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c1ca5a-3321-4a8c-bae8-cb425f6b683a admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:05:33.764315 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1262/2534] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 1 15:05:33 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 15:05:33.769230 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd8a32d-83a1-4fa2-8bbf-8ac9c17570df admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:05:33.779360 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd8a32d-83a1-4fa2-8bbf-8ac9c17570df admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:05:33.783992 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2954f49b-b2c2-4438-befa-e7a9569b499a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:05:33.784737 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2954f49b-b2c2-4438-befa-e7a9569b499a admin admin] REQUEST_METHOD: `PUT` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:05:33.785079 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2954f49b-b2c2-4438-befa-e7a9569b499a admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:05:33.785416 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2954f49b-b2c2-4438-befa-e7a9569b499a admin admin] PATH_INFO: `/v3/projects/9008614ff9044aad8f8566d307daa20c/users/f55b292dcc8e42fd9a84e937b9f67ea4/roles/2a2adb769e55476b82e5014897e0a9c0` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:05:33.800481 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2954f49b-b2c2-4438-befa-e7a9569b499a admin admin] RBAC: Authorizing `identity:create_grant(project_id=9008614ff9044aad8f8566d307daa20c, user_id=f55b292dcc8e42fd9a84e937b9f67ea4, role_id=2a2adb769e55476b82e5014897e0a9c0)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:05:33.803137 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2954f49b-b2c2-4438-befa-e7a9569b499a admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:05:33.817526 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1273/2535] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 1 15:05:33 2022] PUT /identity/v3/projects/9008614ff9044aad8f8566d307daa20c/users/f55b292dcc8e42fd9a84e937b9f67ea4/roles/2a2adb769e55476b82e5014897e0a9c0 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 15:05:33.821581 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c1ca5a-3321-4a8c-bae8-cb425f6b683a admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:05:33.832967 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c1ca5a-3321-4a8c-bae8-cb425f6b683a admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:05:33.892928 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4118018-b7fe-4f84-88c1-b3b1819d8ba8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:05:33.893909 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4118018-b7fe-4f84-88c1-b3b1819d8ba8 admin admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:05:33.894398 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4118018-b7fe-4f84-88c1-b3b1819d8ba8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:05:33.894845 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4118018-b7fe-4f84-88c1-b3b1819d8ba8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:05:33.895982 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4118018-b7fe-4f84-88c1-b3b1819d8ba8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:05:33.898644 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4118018-b7fe-4f84-88c1-b3b1819d8ba8 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:05:33.913830 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1263/2536] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 1 15:05:33 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 15:05:33.915317 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2954f49b-b2c2-4438-befa-e7a9569b499a admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:05:33.926770 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2954f49b-b2c2-4438-befa-e7a9569b499a admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:05:33.931594 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55eba6c4-f880-40cf-91fa-bd576b75385d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:05:33.932239 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55eba6c4-f880-40cf-91fa-bd576b75385d admin admin] REQUEST_METHOD: `PUT` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:05:33.932527 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55eba6c4-f880-40cf-91fa-bd576b75385d admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:05:33.932882 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55eba6c4-f880-40cf-91fa-bd576b75385d admin admin] PATH_INFO: `/v3/projects/9008614ff9044aad8f8566d307daa20c/users/f55b292dcc8e42fd9a84e937b9f67ea4/roles/2a2adb769e55476b82e5014897e0a9c0` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:05:33.934840 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55eba6c4-f880-40cf-91fa-bd576b75385d admin admin] RBAC: Authorizing `identity:create_grant(project_id=9008614ff9044aad8f8566d307daa20c, user_id=f55b292dcc8e42fd9a84e937b9f67ea4, role_id=2a2adb769e55476b82e5014897e0a9c0)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:05:33.937480 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55eba6c4-f880-40cf-91fa-bd576b75385d admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:05:33.950308 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1274/2537] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 1 15:05:33 2022] PUT /identity/v3/projects/9008614ff9044aad8f8566d307daa20c/users/f55b292dcc8e42fd9a84e937b9f67ea4/roles/2a2adb769e55476b82e5014897e0a9c0 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 15:05:33.959777 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f38f4de-b0e0-4eeb-88ce-d78a653644de None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:05:33.960068 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f38f4de-b0e0-4eeb-88ce-d78a653644de None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:05:33.960347 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f38f4de-b0e0-4eeb-88ce-d78a653644de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:05:33.993125 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-6f38f4de-b0e0-4eeb-88ce-d78a653644de None None] MFA Rules not processed for user `f55b292dcc8e42fd9a84e937b9f67ea4`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 15:05:33.994888 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-6f38f4de-b0e0-4eeb-88ce-d78a653644de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:05:34.082941 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1264/2538] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 15:05:33 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 15:05:34.094291 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55eba6c4-f880-40cf-91fa-bd576b75385d admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:05:34.096247 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-55eba6c4-f880-40cf-91fa-bd576b75385d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:05:34.216144 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55eba6c4-f880-40cf-91fa-bd576b75385d admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:05:34.219859 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a278f73-7e40-4402-b91b-6b74cfdd85d4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3b81739e34e4e648f88b6d73fcb5e46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:05:34.220464 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a278f73-7e40-4402-b91b-6b74cfdd85d4 service cinder] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:05:34.220739 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a278f73-7e40-4402-b91b-6b74cfdd85d4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:05:34.221002 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a278f73-7e40-4402-b91b-6b74cfdd85d4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:05:34.231261 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a278f73-7e40-4402-b91b-6b74cfdd85d4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:05:34.232728 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a278f73-7e40-4402-b91b-6b74cfdd85d4 service cinder] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:05:34.246526 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1275/2539] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 15:05:34 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 15:05:34.656129 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f38f4de-b0e0-4eeb-88ce-d78a653644de None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:05:34.665368 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f38f4de-b0e0-4eeb-88ce-d78a653644de None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:05:34.677076 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ec2080-cf6b-406c-ac55-d7d72f5c2690 tempest-VolumeQuotaClassesTest-1927744679 tempest-VolumeQuotaClassesTest-1927744679-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f55b292dcc8e42fd9a84e937b9f67ea4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9008614ff9044aad8f8566d307daa20c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:05:34.677951 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ec2080-cf6b-406c-ac55-d7d72f5c2690 tempest-VolumeQuotaClassesTest-1927744679 tempest-VolumeQuotaClassesTest-1927744679-project-admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:05:34.679366 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ec2080-cf6b-406c-ac55-d7d72f5c2690 tempest-VolumeQuotaClassesTest-1927744679 tempest-VolumeQuotaClassesTest-1927744679-project-admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:05:34.679647 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ec2080-cf6b-406c-ac55-d7d72f5c2690 tempest-VolumeQuotaClassesTest-1927744679 tempest-VolumeQuotaClassesTest-1927744679-project-admin] PATH_INFO: `/v3/domains` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:05:34.680762 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ec2080-cf6b-406c-ac55-d7d72f5c2690 tempest-VolumeQuotaClassesTest-1927744679 tempest-VolumeQuotaClassesTest-1927744679-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81341) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 15:05:34.682048 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ec2080-cf6b-406c-ac55-d7d72f5c2690 tempest-VolumeQuotaClassesTest-1927744679 tempest-VolumeQuotaClassesTest-1927744679-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:05:34.684752 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ec2080-cf6b-406c-ac55-d7d72f5c2690 tempest-VolumeQuotaClassesTest-1927744679 tempest-VolumeQuotaClassesTest-1927744679-project-admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:05:34.703711 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a278f73-7e40-4402-b91b-6b74cfdd85d4 service cinder] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:05:34.704329 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1265/2540] 10.222.0.33 () {58 vars in 1153 bytes} [Thu Sep 1 15:05:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 15:05:34.716606 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a278f73-7e40-4402-b91b-6b74cfdd85d4 service cinder] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:05:34.721858 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee63d9c7-866c-4d56-a76d-dc296992cc2b tempest-VolumeQuotaClassesTest-1927744679 tempest-VolumeQuotaClassesTest-1927744679-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f55b292dcc8e42fd9a84e937b9f67ea4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9008614ff9044aad8f8566d307daa20c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:05:34.722656 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee63d9c7-866c-4d56-a76d-dc296992cc2b tempest-VolumeQuotaClassesTest-1927744679 tempest-VolumeQuotaClassesTest-1927744679-project-admin] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:05:34.723016 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee63d9c7-866c-4d56-a76d-dc296992cc2b tempest-VolumeQuotaClassesTest-1927744679 tempest-VolumeQuotaClassesTest-1927744679-project-admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:05:34.723359 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee63d9c7-866c-4d56-a76d-dc296992cc2b tempest-VolumeQuotaClassesTest-1927744679 tempest-VolumeQuotaClassesTest-1927744679-project-admin] PATH_INFO: `/v3/projects` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:05:34.724589 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee63d9c7-866c-4d56-a76d-dc296992cc2b tempest-VolumeQuotaClassesTest-1927744679 tempest-VolumeQuotaClassesTest-1927744679-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:05:34.726484 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee63d9c7-866c-4d56-a76d-dc296992cc2b tempest-VolumeQuotaClassesTest-1927744679 tempest-VolumeQuotaClassesTest-1927744679-project-admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:05:34.731224 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 15:05:34.794875 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1276/2541] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Sep 1 15:05:34 2022] POST /identity/v3/projects => generated 355 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 15:05:34.799087 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ec2080-cf6b-406c-ac55-d7d72f5c2690 tempest-VolumeQuotaClassesTest-1927744679 tempest-VolumeQuotaClassesTest-1927744679-project-admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:05:34.811128 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ec2080-cf6b-406c-ac55-d7d72f5c2690 tempest-VolumeQuotaClassesTest-1927744679 tempest-VolumeQuotaClassesTest-1927744679-project-admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:05:34.821790 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 15:05:34.846953 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee63d9c7-866c-4d56-a76d-dc296992cc2b tempest-VolumeQuotaClassesTest-1927744679 tempest-VolumeQuotaClassesTest-1927744679-project-admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:05:34.849025 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-ee63d9c7-866c-4d56-a76d-dc296992cc2b tempest-VolumeQuotaClassesTest-1927744679 tempest-VolumeQuotaClassesTest-1927744679-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:05:34.919329 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818e3455-78ce-4647-8cee-da448a817999 tempest-VolumeQuotaClassesTest-1927744679 tempest-VolumeQuotaClassesTest-1927744679-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f55b292dcc8e42fd9a84e937b9f67ea4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9008614ff9044aad8f8566d307daa20c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:05:34.920012 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818e3455-78ce-4647-8cee-da448a817999 tempest-VolumeQuotaClassesTest-1927744679 tempest-VolumeQuotaClassesTest-1927744679-project-admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:05:34.920488 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818e3455-78ce-4647-8cee-da448a817999 tempest-VolumeQuotaClassesTest-1927744679 tempest-VolumeQuotaClassesTest-1927744679-project-admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:05:34.920918 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818e3455-78ce-4647-8cee-da448a817999 tempest-VolumeQuotaClassesTest-1927744679 tempest-VolumeQuotaClassesTest-1927744679-project-admin] PATH_INFO: `/v3/domains` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:05:34.921589 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818e3455-78ce-4647-8cee-da448a817999 tempest-VolumeQuotaClassesTest-1927744679 tempest-VolumeQuotaClassesTest-1927744679-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81341) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 15:05:34.921889 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818e3455-78ce-4647-8cee-da448a817999 tempest-VolumeQuotaClassesTest-1927744679 tempest-VolumeQuotaClassesTest-1927744679-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:05:34.923867 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818e3455-78ce-4647-8cee-da448a817999 tempest-VolumeQuotaClassesTest-1927744679 tempest-VolumeQuotaClassesTest-1927744679-project-admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:05:34.949410 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1266/2542] 10.222.0.33 () {58 vars in 1153 bytes} [Thu Sep 1 15:05:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 15:05:34.950599 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818e3455-78ce-4647-8cee-da448a817999 tempest-VolumeQuotaClassesTest-1927744679 tempest-VolumeQuotaClassesTest-1927744679-project-admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:05:34.958613 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-818e3455-78ce-4647-8cee-da448a817999 tempest-VolumeQuotaClassesTest-1927744679 tempest-VolumeQuotaClassesTest-1927744679-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:05:34.981096 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee63d9c7-866c-4d56-a76d-dc296992cc2b tempest-VolumeQuotaClassesTest-1927744679 tempest-VolumeQuotaClassesTest-1927744679-project-admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:05:34.991039 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ecfa3d-7684-4de5-8938-ef5d4c81eee5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f312c8dcabe48ef9ea453364ade38ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:05:34.991644 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ecfa3d-7684-4de5-8938-ef5d4c81eee5 service placement] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:05:34.991920 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ecfa3d-7684-4de5-8938-ef5d4c81eee5 service placement] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:05:34.992187 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ecfa3d-7684-4de5-8938-ef5d4c81eee5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:05:34.993936 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-b2ecfa3d-7684-4de5-8938-ef5d4c81eee5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:05:35.006495 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ecfa3d-7684-4de5-8938-ef5d4c81eee5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:05:35.008080 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ecfa3d-7684-4de5-8938-ef5d4c81eee5 service placement] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:05:35.017910 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1277/2543] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 15:05:34 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 15:05:35.018833 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ecfa3d-7684-4de5-8938-ef5d4c81eee5 service placement] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:05:35.029464 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ecfa3d-7684-4de5-8938-ef5d4c81eee5 service placement] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:05:35.035529 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040c63d9-6133-4911-8cfe-fb68e8e51169 tempest-VolumeQuotaClassesTest-1927744679 tempest-VolumeQuotaClassesTest-1927744679-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f55b292dcc8e42fd9a84e937b9f67ea4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9008614ff9044aad8f8566d307daa20c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:05:35.036600 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040c63d9-6133-4911-8cfe-fb68e8e51169 tempest-VolumeQuotaClassesTest-1927744679 tempest-VolumeQuotaClassesTest-1927744679-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:05:35.037182 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040c63d9-6133-4911-8cfe-fb68e8e51169 tempest-VolumeQuotaClassesTest-1927744679 tempest-VolumeQuotaClassesTest-1927744679-project-admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:05:35.037894 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040c63d9-6133-4911-8cfe-fb68e8e51169 tempest-VolumeQuotaClassesTest-1927744679 tempest-VolumeQuotaClassesTest-1927744679-project-admin] PATH_INFO: `/v3/projects/d7ffc47245a54d96b25ff019a6f53668` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:05:35.040076 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040c63d9-6133-4911-8cfe-fb68e8e51169 tempest-VolumeQuotaClassesTest-1927744679 tempest-VolumeQuotaClassesTest-1927744679-project-admin] RBAC: Authorizing `identity:delete_project(project_id=d7ffc47245a54d96b25ff019a6f53668)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:05:35.040400 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818e3455-78ce-4647-8cee-da448a817999 tempest-VolumeQuotaClassesTest-1927744679 tempest-VolumeQuotaClassesTest-1927744679-project-admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:05:35.044660 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040c63d9-6133-4911-8cfe-fb68e8e51169 tempest-VolumeQuotaClassesTest-1927744679 tempest-VolumeQuotaClassesTest-1927744679-project-admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:05:35.045959 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7e3f68-fb7f-4a24-bb97-4d62f6503182 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:05:35.046699 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7e3f68-fb7f-4a24-bb97-4d62f6503182 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:05:35.047051 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7e3f68-fb7f-4a24-bb97-4d62f6503182 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:05:35.047398 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7e3f68-fb7f-4a24-bb97-4d62f6503182 admin admin] PATH_INFO: `/v3/users/229e076684dd4bbdacda2231b18f0875` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:05:35.048713 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7e3f68-fb7f-4a24-bb97-4d62f6503182 admin admin] RBAC: Authorizing `identity:delete_user(user_id=229e076684dd4bbdacda2231b18f0875)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:05:35.056708 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7e3f68-fb7f-4a24-bb97-4d62f6503182 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:05:35.095713 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-040c63d9-6133-4911-8cfe-fb68e8e51169 tempest-VolumeQuotaClassesTest-1927744679 tempest-VolumeQuotaClassesTest-1927744679-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd7ffc47245a54d96b25ff019a6f53668', 'request_id': 'req-040c63d9-6133-4911-8cfe-fb68e8e51169'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:05:35.099588 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-8e7e3f68-fb7f-4a24-bb97-4d62f6503182 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '229e076684dd4bbdacda2231b18f0875'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:05:35.102116 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-040c63d9-6133-4911-8cfe-fb68e8e51169 tempest-VolumeQuotaClassesTest-1927744679 tempest-VolumeQuotaClassesTest-1927744679-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd7ffc47245a54d96b25ff019a6f53668', 'request_id': 'req-040c63d9-6133-4911-8cfe-fb68e8e51169'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:05:35.107569 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-8e7e3f68-fb7f-4a24-bb97-4d62f6503182 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '229e076684dd4bbdacda2231b18f0875'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:05:35.108206 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-8e7e3f68-fb7f-4a24-bb97-4d62f6503182 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '229e076684dd4bbdacda2231b18f0875'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:05:35.110414 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-040c63d9-6133-4911-8cfe-fb68e8e51169 tempest-VolumeQuotaClassesTest-1927744679 tempest-VolumeQuotaClassesTest-1927744679-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd7ffc47245a54d96b25ff019a6f53668', 'request_id': 'req-040c63d9-6133-4911-8cfe-fb68e8e51169'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:05:35.121071 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-8e7e3f68-fb7f-4a24-bb97-4d62f6503182 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '229e076684dd4bbdacda2231b18f0875'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:05:35.121503 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-040c63d9-6133-4911-8cfe-fb68e8e51169 tempest-VolumeQuotaClassesTest-1927744679 tempest-VolumeQuotaClassesTest-1927744679-project-admin] The token cache is being invalidate because project d7ffc47245a54d96b25ff019a6f53668 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81342) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 15:05:35.121735 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-8e7e3f68-fb7f-4a24-bb97-4d62f6503182 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '229e076684dd4bbdacda2231b18f0875'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:05:35.121840 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-040c63d9-6133-4911-8cfe-fb68e8e51169 tempest-VolumeQuotaClassesTest-1927744679 tempest-VolumeQuotaClassesTest-1927744679-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:05:35.122517 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-040c63d9-6133-4911-8cfe-fb68e8e51169 tempest-VolumeQuotaClassesTest-1927744679 tempest-VolumeQuotaClassesTest-1927744679-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:05:35.124011 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1278/2544] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Sep 1 15:05:35 2022] DELETE /identity/v3/projects/d7ffc47245a54d96b25ff019a6f53668 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 15:05:35.138474 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1267/2545] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Sep 1 15:05:34 2022] DELETE /identity/v3/users/229e076684dd4bbdacda2231b18f0875 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 15:05:35.331153 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040c63d9-6133-4911-8cfe-fb68e8e51169 tempest-VolumeQuotaClassesTest-1927744679 tempest-VolumeQuotaClassesTest-1927744679-project-admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:05:35.332776 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-040c63d9-6133-4911-8cfe-fb68e8e51169 tempest-VolumeQuotaClassesTest-1927744679 tempest-VolumeQuotaClassesTest-1927744679-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:05:35.349180 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7e3f68-fb7f-4a24-bb97-4d62f6503182 admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:05:35.350665 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-8e7e3f68-fb7f-4a24-bb97-4d62f6503182 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:05:35.381658 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 15:05:35.408718 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 15:05:35.413284 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040c63d9-6133-4911-8cfe-fb68e8e51169 tempest-VolumeQuotaClassesTest-1927744679 tempest-VolumeQuotaClassesTest-1927744679-project-admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:05:35.428954 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7e3f68-fb7f-4a24-bb97-4d62f6503182 admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:05:35.433579 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4eaf26-0604-41ea-a8bb-fc21e69c2730 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:05:35.434205 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4eaf26-0604-41ea-a8bb-fc21e69c2730 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:05:35.434513 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4eaf26-0604-41ea-a8bb-fc21e69c2730 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:05:35.434781 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4eaf26-0604-41ea-a8bb-fc21e69c2730 admin admin] PATH_INFO: `/v3/users/cd922700320343beaffdfeeeb97d7122` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:05:35.435903 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4eaf26-0604-41ea-a8bb-fc21e69c2730 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cd922700320343beaffdfeeeb97d7122)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:05:35.437871 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4eaf26-0604-41ea-a8bb-fc21e69c2730 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:05:35.484423 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9ab9f0-e19b-4117-afc8-003239327209 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:05:35.485197 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9ab9f0-e19b-4117-afc8-003239327209 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:05:35.485609 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9ab9f0-e19b-4117-afc8-003239327209 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:05:35.485988 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9ab9f0-e19b-4117-afc8-003239327209 admin admin] PATH_INFO: `/v3/projects/2f02a6da81c440f9b61e4f2f90f37048` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:05:35.487492 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9ab9f0-e19b-4117-afc8-003239327209 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2f02a6da81c440f9b61e4f2f90f37048)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:05:35.489491 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9ab9f0-e19b-4117-afc8-003239327209 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:05:35.492197 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-1b4eaf26-0604-41ea-a8bb-fc21e69c2730 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cd922700320343beaffdfeeeb97d7122'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:05:35.502675 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-1b4eaf26-0604-41ea-a8bb-fc21e69c2730 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cd922700320343beaffdfeeeb97d7122'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:05:35.503404 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-1b4eaf26-0604-41ea-a8bb-fc21e69c2730 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cd922700320343beaffdfeeeb97d7122'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:05:35.520667 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-1b4eaf26-0604-41ea-a8bb-fc21e69c2730 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cd922700320343beaffdfeeeb97d7122'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:05:35.521479 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-1b4eaf26-0604-41ea-a8bb-fc21e69c2730 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cd922700320343beaffdfeeeb97d7122'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:05:35.537502 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1268/2546] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Sep 1 15:05:35 2022] DELETE /identity/v3/users/cd922700320343beaffdfeeeb97d7122 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 15:05:35.541880 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc96584-f26f-431e-b1ce-7acb713ee28d None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:05:35.543166 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc96584-f26f-431e-b1ce-7acb713ee28d None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:05:35.543448 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc96584-f26f-431e-b1ce-7acb713ee28d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:05:35.545054 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-2f9ab9f0-e19b-4117-afc8-003239327209 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2f02a6da81c440f9b61e4f2f90f37048', 'request_id': 'req-2f9ab9f0-e19b-4117-afc8-003239327209'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:05:35.551797 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-2f9ab9f0-e19b-4117-afc8-003239327209 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2f02a6da81c440f9b61e4f2f90f37048', 'request_id': 'req-2f9ab9f0-e19b-4117-afc8-003239327209'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:05:35.561588 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-2f9ab9f0-e19b-4117-afc8-003239327209 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2f02a6da81c440f9b61e4f2f90f37048', 'request_id': 'req-2f9ab9f0-e19b-4117-afc8-003239327209'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:05:35.563847 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-7bc96584-f26f-431e-b1ce-7acb713ee28d None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 15:05:35.565047 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-7bc96584-f26f-431e-b1ce-7acb713ee28d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:05:35.574726 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-2f9ab9f0-e19b-4117-afc8-003239327209 admin admin] The token cache is being invalidate because project 2f02a6da81c440f9b61e4f2f90f37048 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81342) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 15:05:35.574963 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-2f9ab9f0-e19b-4117-afc8-003239327209 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:05:35.575618 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-2f9ab9f0-e19b-4117-afc8-003239327209 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:05:35.578285 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1279/2547] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Sep 1 15:05:35 2022] DELETE /identity/v3/projects/2f02a6da81c440f9b61e4f2f90f37048 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 15:05:35.582531 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad132ca-386f-43b7-b179-e04131497c98 None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:05:35.582816 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad132ca-386f-43b7-b179-e04131497c98 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:05:35.583094 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad132ca-386f-43b7-b179-e04131497c98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:05:35.605572 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-3ad132ca-386f-43b7-b179-e04131497c98 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 15:05:35.606873 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-3ad132ca-386f-43b7-b179-e04131497c98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:05:35.635078 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1269/2548] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 15:05:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 15:05:35.654522 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc96584-f26f-431e-b1ce-7acb713ee28d None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:05:35.654522 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-7bc96584-f26f-431e-b1ce-7acb713ee28d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:05:35.700196 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1280/2549] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 15:05:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 15:05:35.700403 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 15:05:35.704496 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3969cf-c97e-4c60-8726-54a6b8d435ba None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:05:35.704904 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3969cf-c97e-4c60-8726-54a6b8d435ba None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:05:35.705182 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3969cf-c97e-4c60-8726-54a6b8d435ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:05:35.716873 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc96584-f26f-431e-b1ce-7acb713ee28d None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:05:35.721650 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c0b0bd-f8df-4061-be83-9349aaab3bfd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2790149639aa418bbc4e4982e93eda44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:05:35.722436 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c0b0bd-f8df-4061-be83-9349aaab3bfd service neutron] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:05:35.722842 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c0b0bd-f8df-4061-be83-9349aaab3bfd service neutron] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:05:35.723589 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c0b0bd-f8df-4061-be83-9349aaab3bfd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:05:35.728438 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-6a3969cf-c97e-4c60-8726-54a6b8d435ba None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 15:05:35.729877 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-6a3969cf-c97e-4c60-8726-54a6b8d435ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:05:35.733306 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c0b0bd-f8df-4061-be83-9349aaab3bfd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:05:35.734796 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c0b0bd-f8df-4061-be83-9349aaab3bfd service neutron] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:05:35.738714 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1281/2550] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 15:05:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 15:05:35.742887 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3969cf-c97e-4c60-8726-54a6b8d435ba None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:05:35.744985 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1270/2551] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 15:05:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 15:05:35.753104 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3969cf-c97e-4c60-8726-54a6b8d435ba None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:05:35.758098 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8d786a-6eeb-4145-b6c6-ea35517602ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:05:35.759285 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8d786a-6eeb-4145-b6c6-ea35517602ae admin admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:05:35.759285 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8d786a-6eeb-4145-b6c6-ea35517602ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:05:35.759621 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8d786a-6eeb-4145-b6c6-ea35517602ae admin admin] PATH_INFO: `/v3/domains` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:05:35.760519 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8d786a-6eeb-4145-b6c6-ea35517602ae admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 15:05:35.760905 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8d786a-6eeb-4145-b6c6-ea35517602ae admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:05:35.763555 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8d786a-6eeb-4145-b6c6-ea35517602ae admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:05:35.779781 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1282/2552] 10.222.0.33 () {58 vars in 1153 bytes} [Thu Sep 1 15:05:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 15:05:35.784028 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c0b0bd-f8df-4061-be83-9349aaab3bfd service neutron] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:05:35.796994 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c0b0bd-f8df-4061-be83-9349aaab3bfd service neutron] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:05:35.802498 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe1747b-966c-4a17-b2ab-b75aa8b832c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:05:35.803221 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe1747b-966c-4a17-b2ab-b75aa8b832c7 admin admin] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:05:35.803670 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe1747b-966c-4a17-b2ab-b75aa8b832c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:05:35.804116 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe1747b-966c-4a17-b2ab-b75aa8b832c7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:05:35.804877 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe1747b-966c-4a17-b2ab-b75aa8b832c7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:05:35.807062 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe1747b-966c-4a17-b2ab-b75aa8b832c7 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:05:35.844998 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1271/2553] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Sep 1 15:05:35 2022] POST /identity/v3/projects => generated 367 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 15:05:35.846180 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8d786a-6eeb-4145-b6c6-ea35517602ae admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:05:35.857312 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8d786a-6eeb-4145-b6c6-ea35517602ae admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:05:35.871253 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 15:05:35.929301 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bbb0c9-4afd-4439-8232-a5afb36cfb28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:05:35.930263 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bbb0c9-4afd-4439-8232-a5afb36cfb28 admin admin] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:05:35.930765 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bbb0c9-4afd-4439-8232-a5afb36cfb28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:05:35.931137 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bbb0c9-4afd-4439-8232-a5afb36cfb28 admin admin] PATH_INFO: `/v3/users` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:05:35.932111 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88bbb0c9-4afd-4439-8232-a5afb36cfb28 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:05:35.935004 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88bbb0c9-4afd-4439-8232-a5afb36cfb28 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:05:35.964529 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1283/2554] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Sep 1 15:05:35 2022] POST /identity/v3/users => generated 284 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 15:05:35.965918 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe1747b-966c-4a17-b2ab-b75aa8b832c7 admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:05:35.977304 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe1747b-966c-4a17-b2ab-b75aa8b832c7 admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:05:35.982273 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11fc529-18ce-4b6c-bf1d-1e56da20b2cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:05:35.982899 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11fc529-18ce-4b6c-bf1d-1e56da20b2cd admin admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:05:35.983214 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11fc529-18ce-4b6c-bf1d-1e56da20b2cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:05:35.983580 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11fc529-18ce-4b6c-bf1d-1e56da20b2cd admin admin] PATH_INFO: `/v3/roles` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:05:35.984189 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11fc529-18ce-4b6c-bf1d-1e56da20b2cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:05:35.985852 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11fc529-18ce-4b6c-bf1d-1e56da20b2cd admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:05:35.998641 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1272/2555] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 1 15:05:35 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 15:05:36.002899 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bbb0c9-4afd-4439-8232-a5afb36cfb28 admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:05:36.014587 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bbb0c9-4afd-4439-8232-a5afb36cfb28 admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:05:36.024874 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b9fdef-953e-4a89-9e89-fdb50a1321c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:05:36.025496 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b9fdef-953e-4a89-9e89-fdb50a1321c5 admin admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:05:36.025780 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b9fdef-953e-4a89-9e89-fdb50a1321c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:05:36.026479 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b9fdef-953e-4a89-9e89-fdb50a1321c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:05:36.026761 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b9fdef-953e-4a89-9e89-fdb50a1321c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:05:36.028365 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b9fdef-953e-4a89-9e89-fdb50a1321c5 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:05:36.042284 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1284/2556] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 1 15:05:36 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 15:05:36.045928 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11fc529-18ce-4b6c-bf1d-1e56da20b2cd admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:05:36.054601 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b9fdef-953e-4a89-9e89-fdb50a1321c5 admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:05:36.057008 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-a9b9fdef-953e-4a89-9e89-fdb50a1321c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:05:36.059392 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11fc529-18ce-4b6c-bf1d-1e56da20b2cd admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:05:36.065973 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72160d91-814a-4398-94d2-ebc1811d1fc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:05:36.066671 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72160d91-814a-4398-94d2-ebc1811d1fc5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:05:36.066956 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72160d91-814a-4398-94d2-ebc1811d1fc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:05:36.067296 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72160d91-814a-4398-94d2-ebc1811d1fc5 admin admin] PATH_INFO: `/v3/projects/ae4f1c0791b748209ba7431a4a767882/users/4a91c943542e4c80bcc92035afc415dd/roles/5d1feef0e13d4874928bd86258c4e3b2` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:05:36.076583 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b9fdef-953e-4a89-9e89-fdb50a1321c5 admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:05:36.083515 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f6a063-18e2-4502-bbc6-5a4b23221cf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:05:36.084097 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f6a063-18e2-4502-bbc6-5a4b23221cf6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:05:36.084388 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f6a063-18e2-4502-bbc6-5a4b23221cf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:05:36.084643 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f6a063-18e2-4502-bbc6-5a4b23221cf6 admin admin] PATH_INFO: `/v3/projects/05df60fe39dc4be69ef6a912a44f919d` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:05:36.090520 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f6a063-18e2-4502-bbc6-5a4b23221cf6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=05df60fe39dc4be69ef6a912a44f919d)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:05:36.092446 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f6a063-18e2-4502-bbc6-5a4b23221cf6 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:05:36.102203 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72160d91-814a-4398-94d2-ebc1811d1fc5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae4f1c0791b748209ba7431a4a767882, user_id=4a91c943542e4c80bcc92035afc415dd, role_id=5d1feef0e13d4874928bd86258c4e3b2)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:05:36.104937 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72160d91-814a-4398-94d2-ebc1811d1fc5 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:05:36.121731 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1273/2557] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 1 15:05:36 2022] PUT /identity/v3/projects/ae4f1c0791b748209ba7431a4a767882/users/4a91c943542e4c80bcc92035afc415dd/roles/5d1feef0e13d4874928bd86258c4e3b2 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 15:05:36.133350 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2137b3-f8ec-4e0d-bee4-8841bd09787d None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:05:36.134188 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2137b3-f8ec-4e0d-bee4-8841bd09787d None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:05:36.134496 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2137b3-f8ec-4e0d-bee4-8841bd09787d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:05:36.155477 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-c9f6a063-18e2-4502-bbc6-5a4b23221cf6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '05df60fe39dc4be69ef6a912a44f919d', 'request_id': 'req-c9f6a063-18e2-4502-bbc6-5a4b23221cf6'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:05:36.165647 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-c9f6a063-18e2-4502-bbc6-5a4b23221cf6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '05df60fe39dc4be69ef6a912a44f919d', 'request_id': 'req-c9f6a063-18e2-4502-bbc6-5a4b23221cf6'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:05:36.167939 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-8b2137b3-f8ec-4e0d-bee4-8841bd09787d None None] MFA Rules not processed for user `4a91c943542e4c80bcc92035afc415dd`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 15:05:36.169178 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-8b2137b3-f8ec-4e0d-bee4-8841bd09787d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:05:36.174469 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-c9f6a063-18e2-4502-bbc6-5a4b23221cf6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '05df60fe39dc4be69ef6a912a44f919d', 'request_id': 'req-c9f6a063-18e2-4502-bbc6-5a4b23221cf6'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:05:36.186918 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-c9f6a063-18e2-4502-bbc6-5a4b23221cf6 admin admin] The token cache is being invalidate because project 05df60fe39dc4be69ef6a912a44f919d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81342) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 15:05:36.187205 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-c9f6a063-18e2-4502-bbc6-5a4b23221cf6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:05:36.187881 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-c9f6a063-18e2-4502-bbc6-5a4b23221cf6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:05:36.189476 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1285/2558] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Sep 1 15:05:36 2022] DELETE /identity/v3/projects/05df60fe39dc4be69ef6a912a44f919d => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 15:05:36.193260 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f6a063-18e2-4502-bbc6-5a4b23221cf6 admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:05:36.194651 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-c9f6a063-18e2-4502-bbc6-5a4b23221cf6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:05:36.246875 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1274/2559] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 15:05:36 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 15:05:36.255212 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2137b3-f8ec-4e0d-bee4-8841bd09787d None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:05:36.256978 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-8b2137b3-f8ec-4e0d-bee4-8841bd09787d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:05:36.277090 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f6a063-18e2-4502-bbc6-5a4b23221cf6 admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:05:36.283156 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52dfac84-323c-4ec1-9396-d97dc1332e12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:05:36.283798 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dfac84-323c-4ec1-9396-d97dc1332e12 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:05:36.284085 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dfac84-323c-4ec1-9396-d97dc1332e12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:05:36.284360 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dfac84-323c-4ec1-9396-d97dc1332e12 admin admin] PATH_INFO: `/v3/users/f55b292dcc8e42fd9a84e937b9f67ea4` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:05:36.286184 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52dfac84-323c-4ec1-9396-d97dc1332e12 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f55b292dcc8e42fd9a84e937b9f67ea4)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:05:36.288562 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52dfac84-323c-4ec1-9396-d97dc1332e12 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:05:36.317496 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=87,keystone:INSERT=4,keystone:DELETE=9 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 15:05:36.340927 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2137b3-f8ec-4e0d-bee4-8841bd09787d None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:05:36.344727 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842242d0-6c1a-4279-a355-9124e022e883 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3b81739e34e4e648f88b6d73fcb5e46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:05:36.345327 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842242d0-6c1a-4279-a355-9124e022e883 service cinder] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:05:36.345607 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842242d0-6c1a-4279-a355-9124e022e883 service cinder] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:05:36.345885 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842242d0-6c1a-4279-a355-9124e022e883 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:05:36.352528 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-52dfac84-323c-4ec1-9396-d97dc1332e12 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f55b292dcc8e42fd9a84e937b9f67ea4'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:05:36.357908 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842242d0-6c1a-4279-a355-9124e022e883 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:05:36.359497 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842242d0-6c1a-4279-a355-9124e022e883 service cinder] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:05:36.362051 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-52dfac84-323c-4ec1-9396-d97dc1332e12 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f55b292dcc8e42fd9a84e937b9f67ea4'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:05:36.362869 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-52dfac84-323c-4ec1-9396-d97dc1332e12 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f55b292dcc8e42fd9a84e937b9f67ea4'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:05:36.369400 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1275/2560] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 15:05:36 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 15:05:36.378925 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-52dfac84-323c-4ec1-9396-d97dc1332e12 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f55b292dcc8e42fd9a84e937b9f67ea4'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:05:36.379883 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-52dfac84-323c-4ec1-9396-d97dc1332e12 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f55b292dcc8e42fd9a84e937b9f67ea4'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:05:36.395560 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1286/2561] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Sep 1 15:05:36 2022] DELETE /identity/v3/users/f55b292dcc8e42fd9a84e937b9f67ea4 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 15:05:36.896674 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842242d0-6c1a-4279-a355-9124e022e883 service cinder] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:05:36.918716 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842242d0-6c1a-4279-a355-9124e022e883 service cinder] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:05:36.937831 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 15:05:36.994230 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6472e696-c96c-4abe-8b1c-51b66a049276 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:05:36.995876 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6472e696-c96c-4abe-8b1c-51b66a049276 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:05:36.996432 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6472e696-c96c-4abe-8b1c-51b66a049276 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:05:36.996931 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6472e696-c96c-4abe-8b1c-51b66a049276 admin admin] PATH_INFO: `/v3/projects/9008614ff9044aad8f8566d307daa20c` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:05:37.000161 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6472e696-c96c-4abe-8b1c-51b66a049276 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9008614ff9044aad8f8566d307daa20c)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:05:37.002517 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6472e696-c96c-4abe-8b1c-51b66a049276 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:05:37.065127 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-6472e696-c96c-4abe-8b1c-51b66a049276 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9008614ff9044aad8f8566d307daa20c', 'request_id': 'req-6472e696-c96c-4abe-8b1c-51b66a049276'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:05:37.071782 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-6472e696-c96c-4abe-8b1c-51b66a049276 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9008614ff9044aad8f8566d307daa20c', 'request_id': 'req-6472e696-c96c-4abe-8b1c-51b66a049276'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:05:37.080665 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-6472e696-c96c-4abe-8b1c-51b66a049276 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9008614ff9044aad8f8566d307daa20c', 'request_id': 'req-6472e696-c96c-4abe-8b1c-51b66a049276'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:05:37.092570 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-6472e696-c96c-4abe-8b1c-51b66a049276 admin admin] The token cache is being invalidate because project 9008614ff9044aad8f8566d307daa20c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81341) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 15:05:37.092978 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-6472e696-c96c-4abe-8b1c-51b66a049276 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:05:37.093621 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-6472e696-c96c-4abe-8b1c-51b66a049276 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:05:37.098315 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1276/2562] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Sep 1 15:05:36 2022] DELETE /identity/v3/projects/9008614ff9044aad8f8566d307daa20c => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 15:05:37.100137 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072becdb-9026-4633-9914-6ed4cdfb6bb3 None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:05:37.100427 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072becdb-9026-4633-9914-6ed4cdfb6bb3 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:05:37.100708 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072becdb-9026-4633-9914-6ed4cdfb6bb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:05:37.121804 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-072becdb-9026-4633-9914-6ed4cdfb6bb3 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 15:05:37.123119 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-072becdb-9026-4633-9914-6ed4cdfb6bb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:05:37.192002 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1287/2563] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 15:05:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 15:05:37.199756 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ec39ba-b321-4e2e-bbf0-776ea6acbf58 None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:05:37.200041 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ec39ba-b321-4e2e-bbf0-776ea6acbf58 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:05:37.200317 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ec39ba-b321-4e2e-bbf0-776ea6acbf58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:05:37.220500 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-e6ec39ba-b321-4e2e-bbf0-776ea6acbf58 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 15:05:37.221737 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-e6ec39ba-b321-4e2e-bbf0-776ea6acbf58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:05:37.230726 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1277/2564] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 15:05:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 15:05:37.235127 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072becdb-9026-4633-9914-6ed4cdfb6bb3 None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:05:37.245959 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072becdb-9026-4633-9914-6ed4cdfb6bb3 None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:05:37.250606 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad9dc2d-d044-40b7-a938-a8aaf70aa2b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:05:37.251199 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad9dc2d-d044-40b7-a938-a8aaf70aa2b4 admin admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:05:37.251475 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad9dc2d-d044-40b7-a938-a8aaf70aa2b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:05:37.251744 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad9dc2d-d044-40b7-a938-a8aaf70aa2b4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:05:37.252388 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad9dc2d-d044-40b7-a938-a8aaf70aa2b4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 15:05:37.252685 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad9dc2d-d044-40b7-a938-a8aaf70aa2b4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:05:37.254301 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad9dc2d-d044-40b7-a938-a8aaf70aa2b4 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:05:37.265938 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1288/2565] 10.222.0.33 () {58 vars in 1153 bytes} [Thu Sep 1 15:05:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 15:05:37.270309 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ec39ba-b321-4e2e-bbf0-776ea6acbf58 None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:05:37.279792 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ec39ba-b321-4e2e-bbf0-776ea6acbf58 None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:05:37.284423 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840f3dde-db40-4111-8deb-3ce18d051de3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:05:37.285016 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840f3dde-db40-4111-8deb-3ce18d051de3 admin admin] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:05:37.285292 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840f3dde-db40-4111-8deb-3ce18d051de3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:05:37.285562 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840f3dde-db40-4111-8deb-3ce18d051de3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:05:37.286200 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840f3dde-db40-4111-8deb-3ce18d051de3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:05:37.288182 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840f3dde-db40-4111-8deb-3ce18d051de3 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:05:37.318013 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1278/2566] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Sep 1 15:05:37 2022] POST /identity/v3/projects => generated 401 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 15:05:37.322221 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad9dc2d-d044-40b7-a938-a8aaf70aa2b4 admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:05:37.332439 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad9dc2d-d044-40b7-a938-a8aaf70aa2b4 admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:05:37.395333 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2c43c9-8ec6-42fd-af45-a4cb81eb8a3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:05:37.396009 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2c43c9-8ec6-42fd-af45-a4cb81eb8a3f admin admin] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:05:37.396201 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2c43c9-8ec6-42fd-af45-a4cb81eb8a3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:05:37.396471 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2c43c9-8ec6-42fd-af45-a4cb81eb8a3f admin admin] PATH_INFO: `/v3/users` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:05:37.397167 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2c43c9-8ec6-42fd-af45-a4cb81eb8a3f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:05:37.400426 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2c43c9-8ec6-42fd-af45-a4cb81eb8a3f admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:05:37.430401 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1289/2567] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Sep 1 15:05:37 2022] POST /identity/v3/users => generated 301 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 15:05:37.435238 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840f3dde-db40-4111-8deb-3ce18d051de3 admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:05:37.453904 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840f3dde-db40-4111-8deb-3ce18d051de3 admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:05:37.458706 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bcf881-4936-4832-a7fb-f20f46568d2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:05:37.459518 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bcf881-4936-4832-a7fb-f20f46568d2d admin admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:05:37.460306 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bcf881-4936-4832-a7fb-f20f46568d2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:05:37.460306 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bcf881-4936-4832-a7fb-f20f46568d2d admin admin] PATH_INFO: `/v3/roles` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:05:37.460979 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3bcf881-4936-4832-a7fb-f20f46568d2d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:05:37.462825 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3bcf881-4936-4832-a7fb-f20f46568d2d admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:05:37.475911 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1279/2568] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 1 15:05:37 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 15:05:37.479535 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2c43c9-8ec6-42fd-af45-a4cb81eb8a3f admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:05:37.491017 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2c43c9-8ec6-42fd-af45-a4cb81eb8a3f admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:05:37.496619 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da0dcfd-0409-4bf3-9d6f-e3c21d72559e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:05:37.496944 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da0dcfd-0409-4bf3-9d6f-e3c21d72559e admin admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:05:37.497386 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da0dcfd-0409-4bf3-9d6f-e3c21d72559e admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:05:37.497659 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da0dcfd-0409-4bf3-9d6f-e3c21d72559e admin admin] PATH_INFO: `/v3/roles` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:05:37.498352 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da0dcfd-0409-4bf3-9d6f-e3c21d72559e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:05:37.500091 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da0dcfd-0409-4bf3-9d6f-e3c21d72559e admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:05:37.514285 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1290/2569] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 1 15:05:37 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 15:05:37.515905 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bcf881-4936-4832-a7fb-f20f46568d2d admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:05:37.529653 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bcf881-4936-4832-a7fb-f20f46568d2d admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:05:37.535232 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0140234d-584b-46ba-a36f-0c415ea51b98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:05:37.535902 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0140234d-584b-46ba-a36f-0c415ea51b98 admin admin] REQUEST_METHOD: `PUT` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:05:37.536226 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0140234d-584b-46ba-a36f-0c415ea51b98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:05:37.536576 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0140234d-584b-46ba-a36f-0c415ea51b98 admin admin] PATH_INFO: `/v3/projects/f3ee29ea96124c1d8a3e71d209758f25/users/5db310ac893c4e12ac323fc7ec2c8ead/roles/5d1feef0e13d4874928bd86258c4e3b2` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:05:37.553030 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0140234d-584b-46ba-a36f-0c415ea51b98 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f3ee29ea96124c1d8a3e71d209758f25, user_id=5db310ac893c4e12ac323fc7ec2c8ead, role_id=5d1feef0e13d4874928bd86258c4e3b2)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:05:37.556057 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0140234d-584b-46ba-a36f-0c415ea51b98 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:05:37.570383 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1280/2570] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 1 15:05:37 2022] PUT /identity/v3/projects/f3ee29ea96124c1d8a3e71d209758f25/users/5db310ac893c4e12ac323fc7ec2c8ead/roles/5d1feef0e13d4874928bd86258c4e3b2 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 15:05:37.580917 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2304d3ab-9e1a-4479-aca9-77fdfad0d431 None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:05:37.581259 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2304d3ab-9e1a-4479-aca9-77fdfad0d431 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:05:37.581536 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2304d3ab-9e1a-4479-aca9-77fdfad0d431 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:05:37.589655 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:DELETE=8,keystone:SELECT=88,keystone:INSERT=4 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 15:05:37.613855 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-2304d3ab-9e1a-4479-aca9-77fdfad0d431 None None] MFA Rules not processed for user `5db310ac893c4e12ac323fc7ec2c8ead`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 15:05:37.615101 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-2304d3ab-9e1a-4479-aca9-77fdfad0d431 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:05:37.687743 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1291/2571] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 15:05:37 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 15:05:37.692032 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0140234d-584b-46ba-a36f-0c415ea51b98 admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:05:37.740383 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 15:05:37.760320 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0140234d-584b-46ba-a36f-0c415ea51b98 admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:05:37.765264 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39c81fa-0a61-4b68-92a0-ca623b5bfdf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:05:37.765975 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39c81fa-0a61-4b68-92a0-ca623b5bfdf3 admin admin] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:05:37.766378 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39c81fa-0a61-4b68-92a0-ca623b5bfdf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:05:37.766679 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39c81fa-0a61-4b68-92a0-ca623b5bfdf3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:05:37.767326 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39c81fa-0a61-4b68-92a0-ca623b5bfdf3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:05:37.769307 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39c81fa-0a61-4b68-92a0-ca623b5bfdf3 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:05:37.805113 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1281/2572] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Sep 1 15:05:37 2022] POST /identity/v3/projects => generated 399 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 15:05:37.809332 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2304d3ab-9e1a-4479-aca9-77fdfad0d431 None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:05:37.820016 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2304d3ab-9e1a-4479-aca9-77fdfad0d431 None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:05:37.899825 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e728bd7e-ed7c-4dd7-a7d2-3ca872030336 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:05:37.900511 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e728bd7e-ed7c-4dd7-a7d2-3ca872030336 admin admin] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:05:37.900855 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e728bd7e-ed7c-4dd7-a7d2-3ca872030336 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:05:37.901189 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e728bd7e-ed7c-4dd7-a7d2-3ca872030336 admin admin] PATH_INFO: `/v3/users` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:05:37.902580 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e728bd7e-ed7c-4dd7-a7d2-3ca872030336 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:05:37.906645 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e728bd7e-ed7c-4dd7-a7d2-3ca872030336 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:05:37.915407 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f547b5e0-ee7a-4458-8b3e-97c43652aaca None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:05:37.915887 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f547b5e0-ee7a-4458-8b3e-97c43652aaca None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:05:37.915984 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f547b5e0-ee7a-4458-8b3e-97c43652aaca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:05:37.931961 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1292/2573] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Sep 1 15:05:37 2022] POST /identity/v3/users => generated 306 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 15:05:37.935902 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e728bd7e-ed7c-4dd7-a7d2-3ca872030336 admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:05:37.943084 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-f547b5e0-ee7a-4458-8b3e-97c43652aaca None None] MFA Rules not processed for user `afa6212f31034e27a8d32f114937190e`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 15:05:37.944315 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-f547b5e0-ee7a-4458-8b3e-97c43652aaca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:05:37.946190 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e728bd7e-ed7c-4dd7-a7d2-3ca872030336 admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:05:37.951425 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a93c99-3baf-4a7c-b1e7-ec796fe62689 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:05:37.952018 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a93c99-3baf-4a7c-b1e7-ec796fe62689 admin admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:05:37.952320 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a93c99-3baf-4a7c-b1e7-ec796fe62689 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:05:37.952591 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a93c99-3baf-4a7c-b1e7-ec796fe62689 admin admin] PATH_INFO: `/v3/roles` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:05:37.953358 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a93c99-3baf-4a7c-b1e7-ec796fe62689 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:05:37.954982 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a93c99-3baf-4a7c-b1e7-ec796fe62689 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:05:37.966784 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1293/2574] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 1 15:05:37 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 15:05:37.970802 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a93c99-3baf-4a7c-b1e7-ec796fe62689 admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:05:37.981912 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a93c99-3baf-4a7c-b1e7-ec796fe62689 admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:05:37.989741 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9735ebc4-1a50-4792-89d0-8859774eba20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:05:37.990421 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9735ebc4-1a50-4792-89d0-8859774eba20 admin admin] REQUEST_METHOD: `PUT` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:05:37.990705 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9735ebc4-1a50-4792-89d0-8859774eba20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:05:37.990979 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9735ebc4-1a50-4792-89d0-8859774eba20 admin admin] PATH_INFO: `/v3/projects/82a80180917a488aa5e975fe5df77127/users/00a1a3ed785843d5924c8a003e2939b2/roles/2a2adb769e55476b82e5014897e0a9c0` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:05:38.007163 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9735ebc4-1a50-4792-89d0-8859774eba20 admin admin] RBAC: Authorizing `identity:create_grant(project_id=82a80180917a488aa5e975fe5df77127, user_id=00a1a3ed785843d5924c8a003e2939b2, role_id=2a2adb769e55476b82e5014897e0a9c0)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:05:38.009795 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9735ebc4-1a50-4792-89d0-8859774eba20 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:05:38.013478 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1282/2575] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 15:05:37 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 15:05:38.030216 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1294/2576] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 1 15:05:37 2022] PUT /identity/v3/projects/82a80180917a488aa5e975fe5df77127/users/00a1a3ed785843d5924c8a003e2939b2/roles/2a2adb769e55476b82e5014897e0a9c0 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 15:05:38.031845 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f547b5e0-ee7a-4458-8b3e-97c43652aaca None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:05:38.044706 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f547b5e0-ee7a-4458-8b3e-97c43652aaca None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:05:38.128968 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527bdf11-0d4e-40a4-82cc-cb4c127c7479 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:05:38.129706 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527bdf11-0d4e-40a4-82cc-cb4c127c7479 admin admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:05:38.130045 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527bdf11-0d4e-40a4-82cc-cb4c127c7479 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:05:38.130417 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527bdf11-0d4e-40a4-82cc-cb4c127c7479 admin admin] PATH_INFO: `/v3/roles` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:05:38.131505 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527bdf11-0d4e-40a4-82cc-cb4c127c7479 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:05:38.132931 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527bdf11-0d4e-40a4-82cc-cb4c127c7479 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:05:38.146732 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1283/2577] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 1 15:05:38 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 15:05:38.151404 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9735ebc4-1a50-4792-89d0-8859774eba20 admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:05:38.164566 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9735ebc4-1a50-4792-89d0-8859774eba20 admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:05:38.170955 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a68445b-ae68-41b0-8973-babd896ca60b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:05:38.171832 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a68445b-ae68-41b0-8973-babd896ca60b admin admin] REQUEST_METHOD: `PUT` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:05:38.172179 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a68445b-ae68-41b0-8973-babd896ca60b admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:05:38.172528 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a68445b-ae68-41b0-8973-babd896ca60b admin admin] PATH_INFO: `/v3/projects/82a80180917a488aa5e975fe5df77127/users/00a1a3ed785843d5924c8a003e2939b2/roles/2a2adb769e55476b82e5014897e0a9c0` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:05:38.174599 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a68445b-ae68-41b0-8973-babd896ca60b admin admin] RBAC: Authorizing `identity:create_grant(project_id=82a80180917a488aa5e975fe5df77127, user_id=00a1a3ed785843d5924c8a003e2939b2, role_id=2a2adb769e55476b82e5014897e0a9c0)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:05:38.177498 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a68445b-ae68-41b0-8973-babd896ca60b admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:05:38.202607 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1295/2578] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 1 15:05:38 2022] PUT /identity/v3/projects/82a80180917a488aa5e975fe5df77127/users/00a1a3ed785843d5924c8a003e2939b2/roles/2a2adb769e55476b82e5014897e0a9c0 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 15:05:38.203954 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a870888-39ed-436d-bac3-d1d07c8d1f30 None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:05:38.204268 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a870888-39ed-436d-bac3-d1d07c8d1f30 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:05:38.204548 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a870888-39ed-436d-bac3-d1d07c8d1f30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:05:38.243260 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-7a870888-39ed-436d-bac3-d1d07c8d1f30 None None] MFA Rules not processed for user `00a1a3ed785843d5924c8a003e2939b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 15:05:38.244664 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-7a870888-39ed-436d-bac3-d1d07c8d1f30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:05:38.359062 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 15:05:38.406706 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1284/2579] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 15:05:38 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 15:05:38.419101 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a68445b-ae68-41b0-8973-babd896ca60b admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:05:38.421245 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-5a68445b-ae68-41b0-8973-babd896ca60b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:05:38.459117 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 15:05:38.483713 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a68445b-ae68-41b0-8973-babd896ca60b admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:05:38.487456 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c37ad3d-045b-46aa-8ae9-9584c4e52928 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3b81739e34e4e648f88b6d73fcb5e46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:05:38.488056 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c37ad3d-045b-46aa-8ae9-9584c4e52928 service cinder] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:05:38.488333 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c37ad3d-045b-46aa-8ae9-9584c4e52928 service cinder] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:05:38.488602 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c37ad3d-045b-46aa-8ae9-9584c4e52928 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:05:38.498596 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c37ad3d-045b-46aa-8ae9-9584c4e52928 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:05:38.500030 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c37ad3d-045b-46aa-8ae9-9584c4e52928 service cinder] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:05:38.510809 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1296/2580] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 15:05:38 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 15:05:38.591036 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a870888-39ed-436d-bac3-d1d07c8d1f30 None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:05:38.600725 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a870888-39ed-436d-bac3-d1d07c8d1f30 None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:05:38.604329 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a319c0f5-2e00-4dc4-a78c-0a43cef196ef service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3b81739e34e4e648f88b6d73fcb5e46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:05:38.604953 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a319c0f5-2e00-4dc4-a78c-0a43cef196ef service cinder] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:05:38.605269 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a319c0f5-2e00-4dc4-a78c-0a43cef196ef service cinder] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:05:38.605543 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a319c0f5-2e00-4dc4-a78c-0a43cef196ef service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:05:38.676304 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a319c0f5-2e00-4dc4-a78c-0a43cef196ef service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:05:38.677921 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a319c0f5-2e00-4dc4-a78c-0a43cef196ef service cinder] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:05:38.688042 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1285/2581] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 15:05:38 2022] GET /identity/v3/auth/tokens => generated 3278 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 15:05:42.881478 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c37ad3d-045b-46aa-8ae9-9584c4e52928 service cinder] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:05:42.948157 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c37ad3d-045b-46aa-8ae9-9584c4e52928 service cinder] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:05:42.952656 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a8ae5d-e91e-422e-b41c-fd3b7802554c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:05:42.953333 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a8ae5d-e91e-422e-b41c-fd3b7802554c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:05:42.953607 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a8ae5d-e91e-422e-b41c-fd3b7802554c admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:05:42.953869 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a8ae5d-e91e-422e-b41c-fd3b7802554c admin admin] PATH_INFO: `/v3/users/5db310ac893c4e12ac323fc7ec2c8ead` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:05:42.955022 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a8ae5d-e91e-422e-b41c-fd3b7802554c admin admin] RBAC: Authorizing `identity:delete_user(user_id=5db310ac893c4e12ac323fc7ec2c8ead)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:05:42.956919 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a8ae5d-e91e-422e-b41c-fd3b7802554c admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:05:43.006050 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-17a8ae5d-e91e-422e-b41c-fd3b7802554c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5db310ac893c4e12ac323fc7ec2c8ead'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:05:43.012560 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-17a8ae5d-e91e-422e-b41c-fd3b7802554c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5db310ac893c4e12ac323fc7ec2c8ead'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:05:43.013264 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-17a8ae5d-e91e-422e-b41c-fd3b7802554c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5db310ac893c4e12ac323fc7ec2c8ead'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:05:43.024892 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-17a8ae5d-e91e-422e-b41c-fd3b7802554c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5db310ac893c4e12ac323fc7ec2c8ead'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:05:43.025518 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-17a8ae5d-e91e-422e-b41c-fd3b7802554c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5db310ac893c4e12ac323fc7ec2c8ead'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:05:43.038123 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1297/2582] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Sep 1 15:05:42 2022] DELETE /identity/v3/users/5db310ac893c4e12ac323fc7ec2c8ead => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 15:05:43.043348 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ee4a2c-59a3-444f-bf2c-a9bfc166e4e2 None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:05:43.043835 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ee4a2c-59a3-444f-bf2c-a9bfc166e4e2 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:05:43.043932 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ee4a2c-59a3-444f-bf2c-a9bfc166e4e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:05:43.063152 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-90ee4a2c-59a3-444f-bf2c-a9bfc166e4e2 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 15:05:43.064370 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-90ee4a2c-59a3-444f-bf2c-a9bfc166e4e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:05:43.131005 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1286/2583] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 15:05:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 15:05:43.141440 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a8ae5d-e91e-422e-b41c-fd3b7802554c admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:05:43.142969 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-17a8ae5d-e91e-422e-b41c-fd3b7802554c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:05:43.201240 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a8ae5d-e91e-422e-b41c-fd3b7802554c admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:05:43.204778 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42885fe3-a0d7-4d3d-be72-2eeb930b4756 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2790149639aa418bbc4e4982e93eda44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:05:43.205372 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42885fe3-a0d7-4d3d-be72-2eeb930b4756 service neutron] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:05:43.205648 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42885fe3-a0d7-4d3d-be72-2eeb930b4756 service neutron] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:05:43.205936 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42885fe3-a0d7-4d3d-be72-2eeb930b4756 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:05:43.215191 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42885fe3-a0d7-4d3d-be72-2eeb930b4756 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:05:43.216681 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42885fe3-a0d7-4d3d-be72-2eeb930b4756 service neutron] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:05:43.220511 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 15:05:43.227651 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1298/2584] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 15:05:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 15:05:43.495655 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ee4a2c-59a3-444f-bf2c-a9bfc166e4e2 None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:05:43.497165 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-90ee4a2c-59a3-444f-bf2c-a9bfc166e4e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:05:43.510029 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ee4a2c-59a3-444f-bf2c-a9bfc166e4e2 None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:05:43.515412 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9bebc4-45c8-49ac-9eb3-54c6266e7a26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:05:43.516039 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9bebc4-45c8-49ac-9eb3-54c6266e7a26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:05:43.516317 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9bebc4-45c8-49ac-9eb3-54c6266e7a26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:05:43.516587 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9bebc4-45c8-49ac-9eb3-54c6266e7a26 admin admin] PATH_INFO: `/v3/projects/f3ee29ea96124c1d8a3e71d209758f25` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:05:43.517723 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9bebc4-45c8-49ac-9eb3-54c6266e7a26 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f3ee29ea96124c1d8a3e71d209758f25)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:05:43.519725 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9bebc4-45c8-49ac-9eb3-54c6266e7a26 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:05:43.566425 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-ef9bebc4-45c8-49ac-9eb3-54c6266e7a26 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f3ee29ea96124c1d8a3e71d209758f25', 'request_id': 'req-ef9bebc4-45c8-49ac-9eb3-54c6266e7a26'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:05:43.572434 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-ef9bebc4-45c8-49ac-9eb3-54c6266e7a26 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f3ee29ea96124c1d8a3e71d209758f25', 'request_id': 'req-ef9bebc4-45c8-49ac-9eb3-54c6266e7a26'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:05:43.581377 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-ef9bebc4-45c8-49ac-9eb3-54c6266e7a26 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f3ee29ea96124c1d8a3e71d209758f25', 'request_id': 'req-ef9bebc4-45c8-49ac-9eb3-54c6266e7a26'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:05:43.592673 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-ef9bebc4-45c8-49ac-9eb3-54c6266e7a26 admin admin] The token cache is being invalidate because project f3ee29ea96124c1d8a3e71d209758f25 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81341) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 15:05:43.592958 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-ef9bebc4-45c8-49ac-9eb3-54c6266e7a26 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:05:43.593635 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-ef9bebc4-45c8-49ac-9eb3-54c6266e7a26 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:05:43.595136 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1287/2585] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Sep 1 15:05:43 2022] DELETE /identity/v3/projects/f3ee29ea96124c1d8a3e71d209758f25 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 15:05:43.599162 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42885fe3-a0d7-4d3d-be72-2eeb930b4756 service neutron] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:05:43.608782 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42885fe3-a0d7-4d3d-be72-2eeb930b4756 service neutron] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:05:43.672775 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7358ed3-15ef-49b9-995d-71bb779de167 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:05:43.673351 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7358ed3-15ef-49b9-995d-71bb779de167 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:05:43.673767 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7358ed3-15ef-49b9-995d-71bb779de167 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:05:43.674037 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7358ed3-15ef-49b9-995d-71bb779de167 admin admin] PATH_INFO: `/v3/users/00a1a3ed785843d5924c8a003e2939b2` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:05:43.675363 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7358ed3-15ef-49b9-995d-71bb779de167 admin admin] RBAC: Authorizing `identity:delete_user(user_id=00a1a3ed785843d5924c8a003e2939b2)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:05:43.677287 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7358ed3-15ef-49b9-995d-71bb779de167 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:05:43.725046 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-c7358ed3-15ef-49b9-995d-71bb779de167 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '00a1a3ed785843d5924c8a003e2939b2'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:05:43.733086 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-c7358ed3-15ef-49b9-995d-71bb779de167 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '00a1a3ed785843d5924c8a003e2939b2'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:05:43.733754 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-c7358ed3-15ef-49b9-995d-71bb779de167 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '00a1a3ed785843d5924c8a003e2939b2'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:05:43.753115 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-c7358ed3-15ef-49b9-995d-71bb779de167 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '00a1a3ed785843d5924c8a003e2939b2'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:05:43.753795 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-c7358ed3-15ef-49b9-995d-71bb779de167 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '00a1a3ed785843d5924c8a003e2939b2'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:05:43.766496 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1299/2586] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Sep 1 15:05:43 2022] DELETE /identity/v3/users/00a1a3ed785843d5924c8a003e2939b2 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 15:05:44.034369 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9bebc4-45c8-49ac-9eb3-54c6266e7a26 admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:05:44.035828 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-ef9bebc4-45c8-49ac-9eb3-54c6266e7a26 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:05:44.067613 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9bebc4-45c8-49ac-9eb3-54c6266e7a26 admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:05:44.082303 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 15:05:44.133702 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd842b63-3fdb-440a-9931-77ae42b6e4ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:05:44.134479 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd842b63-3fdb-440a-9931-77ae42b6e4ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:05:44.134763 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd842b63-3fdb-440a-9931-77ae42b6e4ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:05:44.135036 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd842b63-3fdb-440a-9931-77ae42b6e4ca admin admin] PATH_INFO: `/v3/projects/82a80180917a488aa5e975fe5df77127` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:05:44.136213 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd842b63-3fdb-440a-9931-77ae42b6e4ca admin admin] RBAC: Authorizing `identity:delete_project(project_id=82a80180917a488aa5e975fe5df77127)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:05:44.138124 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd842b63-3fdb-440a-9931-77ae42b6e4ca admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:05:44.186738 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-cd842b63-3fdb-440a-9931-77ae42b6e4ca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '82a80180917a488aa5e975fe5df77127', 'request_id': 'req-cd842b63-3fdb-440a-9931-77ae42b6e4ca'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:05:44.192682 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-cd842b63-3fdb-440a-9931-77ae42b6e4ca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '82a80180917a488aa5e975fe5df77127', 'request_id': 'req-cd842b63-3fdb-440a-9931-77ae42b6e4ca'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:05:44.200034 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-cd842b63-3fdb-440a-9931-77ae42b6e4ca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '82a80180917a488aa5e975fe5df77127', 'request_id': 'req-cd842b63-3fdb-440a-9931-77ae42b6e4ca'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:05:44.210110 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-cd842b63-3fdb-440a-9931-77ae42b6e4ca admin admin] The token cache is being invalidate because project 82a80180917a488aa5e975fe5df77127 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81341) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 15:05:44.210438 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-cd842b63-3fdb-440a-9931-77ae42b6e4ca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:05:44.211065 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-cd842b63-3fdb-440a-9931-77ae42b6e4ca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:05:44.212531 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1288/2587] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Sep 1 15:05:44 2022] DELETE /identity/v3/projects/82a80180917a488aa5e975fe5df77127 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 15:05:44.217861 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabfb0fe-4652-4d25-900a-48424b2a734d None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:05:44.218425 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabfb0fe-4652-4d25-900a-48424b2a734d None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:05:44.218526 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabfb0fe-4652-4d25-900a-48424b2a734d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:05:44.239284 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-aabfb0fe-4652-4d25-900a-48424b2a734d None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 15:05:44.240524 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-aabfb0fe-4652-4d25-900a-48424b2a734d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:05:44.305702 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1300/2588] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 15:05:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 15:05:44.313817 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea99ca3-f601-4a2f-b015-5d234e0aac0d None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:05:44.314213 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea99ca3-f601-4a2f-b015-5d234e0aac0d None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:05:44.314634 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea99ca3-f601-4a2f-b015-5d234e0aac0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:05:44.334882 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-9ea99ca3-f601-4a2f-b015-5d234e0aac0d None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 15:05:44.336091 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-9ea99ca3-f601-4a2f-b015-5d234e0aac0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:05:44.344642 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1289/2589] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 15:05:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 15:05:44.348943 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabfb0fe-4652-4d25-900a-48424b2a734d None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:05:44.359137 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabfb0fe-4652-4d25-900a-48424b2a734d None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:05:44.363675 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a328a10d-a92e-4699-bf2b-f1ac7dbe82c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:05:44.364260 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a328a10d-a92e-4699-bf2b-f1ac7dbe82c0 admin admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:05:44.364533 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a328a10d-a92e-4699-bf2b-f1ac7dbe82c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:05:44.364797 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a328a10d-a92e-4699-bf2b-f1ac7dbe82c0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:05:44.365432 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a328a10d-a92e-4699-bf2b-f1ac7dbe82c0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 15:05:44.365729 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a328a10d-a92e-4699-bf2b-f1ac7dbe82c0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:05:44.367386 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a328a10d-a92e-4699-bf2b-f1ac7dbe82c0 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:05:44.378763 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1301/2590] 10.222.0.33 () {58 vars in 1153 bytes} [Thu Sep 1 15:05:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 15:05:44.382818 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea99ca3-f601-4a2f-b015-5d234e0aac0d None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:05:44.391952 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea99ca3-f601-4a2f-b015-5d234e0aac0d None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:05:44.396593 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01df95cb-c26b-433b-a061-11fdefff1fe6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:05:44.397175 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01df95cb-c26b-433b-a061-11fdefff1fe6 admin admin] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:05:44.397475 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01df95cb-c26b-433b-a061-11fdefff1fe6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:05:44.397744 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01df95cb-c26b-433b-a061-11fdefff1fe6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:05:44.398419 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01df95cb-c26b-433b-a061-11fdefff1fe6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:05:44.400308 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01df95cb-c26b-433b-a061-11fdefff1fe6 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:05:44.427949 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1290/2591] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Sep 1 15:05:44 2022] POST /identity/v3/projects => generated 373 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 15:05:44.432027 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a328a10d-a92e-4699-bf2b-f1ac7dbe82c0 admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:05:44.438148 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 15:05:44.441806 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a328a10d-a92e-4699-bf2b-f1ac7dbe82c0 admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:05:44.508783 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d6378a-2885-4170-a02d-7e9c03d7d573 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:05:44.509663 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d6378a-2885-4170-a02d-7e9c03d7d573 admin admin] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:05:44.509945 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d6378a-2885-4170-a02d-7e9c03d7d573 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:05:44.510219 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d6378a-2885-4170-a02d-7e9c03d7d573 admin admin] PATH_INFO: `/v3/users` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:05:44.511085 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d6378a-2885-4170-a02d-7e9c03d7d573 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:05:44.513064 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d6378a-2885-4170-a02d-7e9c03d7d573 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:05:44.534536 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1302/2592] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Sep 1 15:05:44 2022] POST /identity/v3/users => generated 287 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 15:05:44.538801 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01df95cb-c26b-433b-a061-11fdefff1fe6 admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:05:44.550572 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01df95cb-c26b-433b-a061-11fdefff1fe6 admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:05:44.555204 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847a3712-2d77-4a1e-978a-51b388bb87c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:05:44.555837 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847a3712-2d77-4a1e-978a-51b388bb87c0 admin admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:05:44.556063 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847a3712-2d77-4a1e-978a-51b388bb87c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:05:44.556478 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847a3712-2d77-4a1e-978a-51b388bb87c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:05:44.557115 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847a3712-2d77-4a1e-978a-51b388bb87c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:05:44.558770 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847a3712-2d77-4a1e-978a-51b388bb87c0 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:05:44.568484 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1291/2593] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 1 15:05:44 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 15:05:44.572506 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d6378a-2885-4170-a02d-7e9c03d7d573 admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:05:44.582723 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d6378a-2885-4170-a02d-7e9c03d7d573 admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:05:44.587321 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7844ef5e-f0bf-4e5a-8536-5183ac73ebeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:05:44.587908 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7844ef5e-f0bf-4e5a-8536-5183ac73ebeb admin admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:05:44.588188 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7844ef5e-f0bf-4e5a-8536-5183ac73ebeb admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:05:44.588458 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7844ef5e-f0bf-4e5a-8536-5183ac73ebeb admin admin] PATH_INFO: `/v3/roles` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:05:44.589768 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7844ef5e-f0bf-4e5a-8536-5183ac73ebeb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:05:44.591391 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7844ef5e-f0bf-4e5a-8536-5183ac73ebeb admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:05:44.601706 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1303/2594] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 1 15:05:44 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 15:05:44.605779 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847a3712-2d77-4a1e-978a-51b388bb87c0 admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:05:44.615733 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847a3712-2d77-4a1e-978a-51b388bb87c0 admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:05:44.620303 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bba858-30f0-41dc-aa39-f0b339e62223 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:05:44.620931 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bba858-30f0-41dc-aa39-f0b339e62223 admin admin] REQUEST_METHOD: `PUT` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:05:44.621238 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bba858-30f0-41dc-aa39-f0b339e62223 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:05:44.621506 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bba858-30f0-41dc-aa39-f0b339e62223 admin admin] PATH_INFO: `/v3/projects/9eb2f6885d9944bf90f2aa789a990749/users/b6682ccfe402445b82919b1530ac8c10/roles/5d1feef0e13d4874928bd86258c4e3b2` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:05:44.635459 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bba858-30f0-41dc-aa39-f0b339e62223 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9eb2f6885d9944bf90f2aa789a990749, user_id=b6682ccfe402445b82919b1530ac8c10, role_id=5d1feef0e13d4874928bd86258c4e3b2)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:05:44.638689 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bba858-30f0-41dc-aa39-f0b339e62223 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:05:44.651202 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1292/2595] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 1 15:05:44 2022] PUT /identity/v3/projects/9eb2f6885d9944bf90f2aa789a990749/users/b6682ccfe402445b82919b1530ac8c10/roles/5d1feef0e13d4874928bd86258c4e3b2 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 15:05:44.656929 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c9d3f6-6524-4015-b8df-7c31418c4aed None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:05:44.657406 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c9d3f6-6524-4015-b8df-7c31418c4aed None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:05:44.657846 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c9d3f6-6524-4015-b8df-7c31418c4aed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:05:44.676818 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-00c9d3f6-6524-4015-b8df-7c31418c4aed None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 15:05:44.678010 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-00c9d3f6-6524-4015-b8df-7c31418c4aed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:05:44.734615 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1304/2596] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 15:05:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 15:05:44.744268 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bba858-30f0-41dc-aa39-f0b339e62223 admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:05:44.745991 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-49bba858-30f0-41dc-aa39-f0b339e62223 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:05:44.795208 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 15:05:44.807588 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bba858-30f0-41dc-aa39-f0b339e62223 admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:05:44.811792 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97212554-c6ca-4824-bbf3-ad81748f3b61 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2790149639aa418bbc4e4982e93eda44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:05:44.812404 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97212554-c6ca-4824-bbf3-ad81748f3b61 service neutron] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:05:44.812679 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97212554-c6ca-4824-bbf3-ad81748f3b61 service neutron] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:05:44.812949 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97212554-c6ca-4824-bbf3-ad81748f3b61 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:05:44.822418 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97212554-c6ca-4824-bbf3-ad81748f3b61 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:05:44.824449 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97212554-c6ca-4824-bbf3-ad81748f3b61 service neutron] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:05:44.834103 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1293/2597] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 15:05:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 15:05:45.158902 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c9d3f6-6524-4015-b8df-7c31418c4aed None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:05:45.160626 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-00c9d3f6-6524-4015-b8df-7c31418c4aed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:05:45.176779 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c9d3f6-6524-4015-b8df-7c31418c4aed None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:05:45.181549 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73e50a0-1a08-420f-b6b1-611c82865d2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:05:45.182278 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73e50a0-1a08-420f-b6b1-611c82865d2a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:05:45.183285 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73e50a0-1a08-420f-b6b1-611c82865d2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:05:45.183603 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73e50a0-1a08-420f-b6b1-611c82865d2a admin admin] PATH_INFO: `/v3/users/4a91c943542e4c80bcc92035afc415dd` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:05:45.184885 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73e50a0-1a08-420f-b6b1-611c82865d2a admin admin] RBAC: Authorizing `identity:delete_user(user_id=4a91c943542e4c80bcc92035afc415dd)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:05:45.187092 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73e50a0-1a08-420f-b6b1-611c82865d2a admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:05:45.241085 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-e73e50a0-1a08-420f-b6b1-611c82865d2a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4a91c943542e4c80bcc92035afc415dd'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:05:45.250363 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-e73e50a0-1a08-420f-b6b1-611c82865d2a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4a91c943542e4c80bcc92035afc415dd'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:05:45.251013 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-e73e50a0-1a08-420f-b6b1-611c82865d2a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4a91c943542e4c80bcc92035afc415dd'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:05:45.264669 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-e73e50a0-1a08-420f-b6b1-611c82865d2a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4a91c943542e4c80bcc92035afc415dd'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:05:45.265345 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-e73e50a0-1a08-420f-b6b1-611c82865d2a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4a91c943542e4c80bcc92035afc415dd'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:05:45.270595 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 15:05:45.279886 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1305/2598] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Sep 1 15:05:45 2022] DELETE /identity/v3/users/4a91c943542e4c80bcc92035afc415dd => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 15:05:45.284228 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6788cf3-03f6-43d0-88ea-034603df9c56 None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:05:45.284711 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6788cf3-03f6-43d0-88ea-034603df9c56 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:05:45.284820 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6788cf3-03f6-43d0-88ea-034603df9c56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:05:45.304688 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-c6788cf3-03f6-43d0-88ea-034603df9c56 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 15:05:45.305934 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-c6788cf3-03f6-43d0-88ea-034603df9c56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:05:45.367429 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1294/2599] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 15:05:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 15:05:45.377518 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73e50a0-1a08-420f-b6b1-611c82865d2a admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:05:45.379021 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-e73e50a0-1a08-420f-b6b1-611c82865d2a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:05:45.438162 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73e50a0-1a08-420f-b6b1-611c82865d2a admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:05:45.442005 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32c49d6-7d40-4e77-a412-04d4599a77fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2790149639aa418bbc4e4982e93eda44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:05:45.442680 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32c49d6-7d40-4e77-a412-04d4599a77fe service neutron] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:05:45.442989 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32c49d6-7d40-4e77-a412-04d4599a77fe service neutron] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:05:45.443468 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32c49d6-7d40-4e77-a412-04d4599a77fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:05:45.452893 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32c49d6-7d40-4e77-a412-04d4599a77fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:05:45.454471 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32c49d6-7d40-4e77-a412-04d4599a77fe service neutron] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:05:45.465438 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1306/2600] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 15:05:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 15:05:45.489422 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec634c58-9119-4f83-adf8-24ed08e8c403 None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:05:45.489720 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec634c58-9119-4f83-adf8-24ed08e8c403 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:05:45.490039 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec634c58-9119-4f83-adf8-24ed08e8c403 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:05:45.521667 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-ec634c58-9119-4f83-adf8-24ed08e8c403 None None] MFA Rules not processed for user `b6682ccfe402445b82919b1530ac8c10`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 15:05:45.523110 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-ec634c58-9119-4f83-adf8-24ed08e8c403 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:05:45.606978 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1295/2601] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 15:05:45 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 15:05:45.611433 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32c49d6-7d40-4e77-a412-04d4599a77fe service neutron] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:05:45.613249 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-f32c49d6-7d40-4e77-a412-04d4599a77fe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:05:45.629691 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32c49d6-7d40-4e77-a412-04d4599a77fe service neutron] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:05:45.654308 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00bbd9b-6cc0-4a3e-8a06-6f3cc2826ad2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:05:45.655798 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00bbd9b-6cc0-4a3e-8a06-6f3cc2826ad2 admin admin] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:05:45.656283 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00bbd9b-6cc0-4a3e-8a06-6f3cc2826ad2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:05:45.656683 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00bbd9b-6cc0-4a3e-8a06-6f3cc2826ad2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:05:45.657368 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00bbd9b-6cc0-4a3e-8a06-6f3cc2826ad2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:05:45.659825 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00bbd9b-6cc0-4a3e-8a06-6f3cc2826ad2 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:05:45.693229 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1307/2602] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Sep 1 15:05:45 2022] POST /identity/v3/projects => generated 369 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 15:05:45.697188 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec634c58-9119-4f83-adf8-24ed08e8c403 None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:05:45.708604 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec634c58-9119-4f83-adf8-24ed08e8c403 None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:05:45.768172 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00bbd9b-6cc0-4a3e-8a06-6f3cc2826ad2 admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:05:45.769146 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9cea58-c14a-4556-bfa5-6f2c9da4d832 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:05:45.769826 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9cea58-c14a-4556-bfa5-6f2c9da4d832 admin admin] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:05:45.770184 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9cea58-c14a-4556-bfa5-6f2c9da4d832 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:05:45.770461 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-c00bbd9b-6cc0-4a3e-8a06-6f3cc2826ad2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:05:45.770574 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9cea58-c14a-4556-bfa5-6f2c9da4d832 admin admin] PATH_INFO: `/v3/users` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:05:45.771292 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9cea58-c14a-4556-bfa5-6f2c9da4d832 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:05:45.773387 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9cea58-c14a-4556-bfa5-6f2c9da4d832 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:05:45.785120 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00bbd9b-6cc0-4a3e-8a06-6f3cc2826ad2 admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:05:45.792834 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8026ecab-7ab0-4c6e-aadd-39bb80dcc66b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:05:45.793808 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8026ecab-7ab0-4c6e-aadd-39bb80dcc66b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:05:45.794100 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8026ecab-7ab0-4c6e-aadd-39bb80dcc66b admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:05:45.794547 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8026ecab-7ab0-4c6e-aadd-39bb80dcc66b admin admin] PATH_INFO: `/v3/projects/ae4f1c0791b748209ba7431a4a767882` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:05:45.796030 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8026ecab-7ab0-4c6e-aadd-39bb80dcc66b admin admin] RBAC: Authorizing `identity:delete_project(project_id=ae4f1c0791b748209ba7431a4a767882)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:05:45.797742 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1296/2603] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Sep 1 15:05:45 2022] POST /identity/v3/users => generated 291 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 15:05:45.798046 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8026ecab-7ab0-4c6e-aadd-39bb80dcc66b admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:05:45.801590 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9cea58-c14a-4556-bfa5-6f2c9da4d832 admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:05:45.811855 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9cea58-c14a-4556-bfa5-6f2c9da4d832 admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:05:45.816553 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3a2329-7990-4c6d-8a1b-274664096c8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:05:45.817137 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3a2329-7990-4c6d-8a1b-274664096c8a admin admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:05:45.817413 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3a2329-7990-4c6d-8a1b-274664096c8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:05:45.817678 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3a2329-7990-4c6d-8a1b-274664096c8a admin admin] PATH_INFO: `/v3/roles` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:05:45.818382 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3a2329-7990-4c6d-8a1b-274664096c8a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:05:45.820467 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3a2329-7990-4c6d-8a1b-274664096c8a admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:05:45.821763 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 15:05:45.834612 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1297/2604] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 1 15:05:45 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 15:05:45.835639 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3a2329-7990-4c6d-8a1b-274664096c8a admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:05:45.847288 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3a2329-7990-4c6d-8a1b-274664096c8a admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:05:45.850359 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-8026ecab-7ab0-4c6e-aadd-39bb80dcc66b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ae4f1c0791b748209ba7431a4a767882', 'request_id': 'req-8026ecab-7ab0-4c6e-aadd-39bb80dcc66b'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:05:45.858274 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-8026ecab-7ab0-4c6e-aadd-39bb80dcc66b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ae4f1c0791b748209ba7431a4a767882', 'request_id': 'req-8026ecab-7ab0-4c6e-aadd-39bb80dcc66b'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:05:45.865845 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-8026ecab-7ab0-4c6e-aadd-39bb80dcc66b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ae4f1c0791b748209ba7431a4a767882', 'request_id': 'req-8026ecab-7ab0-4c6e-aadd-39bb80dcc66b'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:05:45.876590 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-8026ecab-7ab0-4c6e-aadd-39bb80dcc66b admin admin] The token cache is being invalidate because project ae4f1c0791b748209ba7431a4a767882 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81342) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 15:05:45.876879 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-8026ecab-7ab0-4c6e-aadd-39bb80dcc66b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:05:45.877925 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-8026ecab-7ab0-4c6e-aadd-39bb80dcc66b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:05:45.879474 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1308/2605] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Sep 1 15:05:45 2022] DELETE /identity/v3/projects/ae4f1c0791b748209ba7431a4a767882 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 15:05:45.884028 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e165c5bc-a1b3-4852-b7a8-3c73aa77d04c None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:05:45.884315 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e165c5bc-a1b3-4852-b7a8-3c73aa77d04c None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:05:45.884594 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e165c5bc-a1b3-4852-b7a8-3c73aa77d04c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:05:45.903778 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-e165c5bc-a1b3-4852-b7a8-3c73aa77d04c None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 15:05:45.904992 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-e165c5bc-a1b3-4852-b7a8-3c73aa77d04c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:05:45.906179 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471cef13-15a1-4bb7-b50b-fa32eb9f09d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:05:45.906870 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471cef13-15a1-4bb7-b50b-fa32eb9f09d1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:05:45.907146 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471cef13-15a1-4bb7-b50b-fa32eb9f09d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:05:45.907414 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471cef13-15a1-4bb7-b50b-fa32eb9f09d1 admin admin] PATH_INFO: `/v3/projects/6aed9bea45b243d093faa957a8087fed/users/497118c4126444398c23cd951eed7086/roles/2a2adb769e55476b82e5014897e0a9c0` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:05:45.912873 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1309/2606] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 15:05:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 15:05:45.922618 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be01967c-cafc-499d-8fc2-a20f81c3ded0 None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:05:45.922901 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be01967c-cafc-499d-8fc2-a20f81c3ded0 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:05:45.923085 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471cef13-15a1-4bb7-b50b-fa32eb9f09d1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6aed9bea45b243d093faa957a8087fed, user_id=497118c4126444398c23cd951eed7086, role_id=2a2adb769e55476b82e5014897e0a9c0)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:05:45.923218 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be01967c-cafc-499d-8fc2-a20f81c3ded0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:05:45.925907 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471cef13-15a1-4bb7-b50b-fa32eb9f09d1 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:05:45.941102 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1298/2607] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 1 15:05:45 2022] PUT /identity/v3/projects/6aed9bea45b243d093faa957a8087fed/users/497118c4126444398c23cd951eed7086/roles/2a2adb769e55476b82e5014897e0a9c0 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 15:05:45.945040 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471cef13-15a1-4bb7-b50b-fa32eb9f09d1 admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:05:45.945385 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-be01967c-cafc-499d-8fc2-a20f81c3ded0 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 15:05:45.946846 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-be01967c-cafc-499d-8fc2-a20f81c3ded0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:05:45.947105 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-471cef13-15a1-4bb7-b50b-fa32eb9f09d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:05:46.006168 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 15:05:46.019532 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471cef13-15a1-4bb7-b50b-fa32eb9f09d1 admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:05:46.024782 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1310/2608] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 15:05:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 15:05:46.028506 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efde7984-0ed8-4a6f-a132-b9fcb0915b17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:05:46.028506 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efde7984-0ed8-4a6f-a132-b9fcb0915b17 admin admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:05:46.028506 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efde7984-0ed8-4a6f-a132-b9fcb0915b17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:05:46.028506 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efde7984-0ed8-4a6f-a132-b9fcb0915b17 admin admin] PATH_INFO: `/v3/roles` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:05:46.028942 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efde7984-0ed8-4a6f-a132-b9fcb0915b17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:05:46.029766 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be01967c-cafc-499d-8fc2-a20f81c3ded0 None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:05:46.029888 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efde7984-0ed8-4a6f-a132-b9fcb0915b17 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:05:46.040504 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1299/2609] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 1 15:05:45 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 15:05:46.040854 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be01967c-cafc-499d-8fc2-a20f81c3ded0 None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:05:46.044851 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efde7984-0ed8-4a6f-a132-b9fcb0915b17 admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:05:46.048951 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0d93cd-ebd7-4c9a-9186-39e30916c472 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:05:46.049652 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0d93cd-ebd7-4c9a-9186-39e30916c472 admin admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:05:46.050011 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0d93cd-ebd7-4c9a-9186-39e30916c472 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:05:46.050390 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0d93cd-ebd7-4c9a-9186-39e30916c472 admin admin] PATH_INFO: `/v3/domains` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:05:46.051528 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0d93cd-ebd7-4c9a-9186-39e30916c472 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 15:05:46.051528 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0d93cd-ebd7-4c9a-9186-39e30916c472 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:05:46.053012 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0d93cd-ebd7-4c9a-9186-39e30916c472 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:05:46.055431 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efde7984-0ed8-4a6f-a132-b9fcb0915b17 admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:05:46.060665 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95387ba0-9a70-408d-862e-668256207c8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:05:46.061674 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95387ba0-9a70-408d-862e-668256207c8e admin admin] REQUEST_METHOD: `PUT` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:05:46.061674 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95387ba0-9a70-408d-862e-668256207c8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:05:46.061901 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95387ba0-9a70-408d-862e-668256207c8e admin admin] PATH_INFO: `/v3/projects/6aed9bea45b243d093faa957a8087fed/users/497118c4126444398c23cd951eed7086/roles/2a2adb769e55476b82e5014897e0a9c0` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:05:46.064097 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95387ba0-9a70-408d-862e-668256207c8e admin admin] RBAC: Authorizing `identity:create_grant(project_id=6aed9bea45b243d093faa957a8087fed, user_id=497118c4126444398c23cd951eed7086, role_id=2a2adb769e55476b82e5014897e0a9c0)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:05:46.066626 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1311/2610] 10.222.0.33 () {58 vars in 1153 bytes} [Thu Sep 1 15:05:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 15:05:46.066813 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95387ba0-9a70-408d-862e-668256207c8e admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:05:46.070854 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0d93cd-ebd7-4c9a-9186-39e30916c472 admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:05:46.079211 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1300/2611] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 1 15:05:46 2022] PUT /identity/v3/projects/6aed9bea45b243d093faa957a8087fed/users/497118c4126444398c23cd951eed7086/roles/2a2adb769e55476b82e5014897e0a9c0 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 15:05:46.081645 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0d93cd-ebd7-4c9a-9186-39e30916c472 admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:05:46.141870 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ddbf5c-a66f-4dbf-b0b9-450802b7c79d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:05:46.142587 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ddbf5c-a66f-4dbf-b0b9-450802b7c79d admin admin] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:05:46.142929 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ddbf5c-a66f-4dbf-b0b9-450802b7c79d admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:05:46.143262 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ddbf5c-a66f-4dbf-b0b9-450802b7c79d admin admin] PATH_INFO: `/v3/projects` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:05:46.143944 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ddbf5c-a66f-4dbf-b0b9-450802b7c79d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:05:46.145847 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ddbf5c-a66f-4dbf-b0b9-450802b7c79d admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:05:46.175440 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1312/2612] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Sep 1 15:05:46 2022] POST /identity/v3/projects => generated 367 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 15:05:46.179738 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95387ba0-9a70-408d-862e-668256207c8e admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:05:46.191653 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95387ba0-9a70-408d-862e-668256207c8e admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:05:46.252478 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf7aa3f-d31c-441d-9d83-a41f9f18b9e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:05:46.253176 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf7aa3f-d31c-441d-9d83-a41f9f18b9e6 admin admin] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:05:46.253354 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf7aa3f-d31c-441d-9d83-a41f9f18b9e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:05:46.253636 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf7aa3f-d31c-441d-9d83-a41f9f18b9e6 admin admin] PATH_INFO: `/v3/users` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:05:46.254337 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf7aa3f-d31c-441d-9d83-a41f9f18b9e6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:05:46.256279 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf7aa3f-d31c-441d-9d83-a41f9f18b9e6 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:05:46.266780 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 15:05:46.284450 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1301/2613] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Sep 1 15:05:46 2022] POST /identity/v3/users => generated 284 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 15:05:46.288544 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ddbf5c-a66f-4dbf-b0b9-450802b7c79d admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:05:46.299323 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ddbf5c-a66f-4dbf-b0b9-450802b7c79d admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:05:46.305613 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02398c00-e44b-4511-92db-a4fd3de01007 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:05:46.306433 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02398c00-e44b-4511-92db-a4fd3de01007 admin admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:05:46.306712 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02398c00-e44b-4511-92db-a4fd3de01007 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:05:46.307516 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02398c00-e44b-4511-92db-a4fd3de01007 admin admin] PATH_INFO: `/v3/roles` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:05:46.307967 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02398c00-e44b-4511-92db-a4fd3de01007 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:05:46.310121 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02398c00-e44b-4511-92db-a4fd3de01007 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:05:46.321304 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1313/2614] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 1 15:05:46 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 15:05:46.326060 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf7aa3f-d31c-441d-9d83-a41f9f18b9e6 admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:05:46.336297 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf7aa3f-d31c-441d-9d83-a41f9f18b9e6 admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:05:46.342094 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2404f963-672b-4142-907a-b6fac2d98182 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:05:46.342798 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2404f963-672b-4142-907a-b6fac2d98182 admin admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:05:46.343079 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2404f963-672b-4142-907a-b6fac2d98182 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:05:46.343353 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2404f963-672b-4142-907a-b6fac2d98182 admin admin] PATH_INFO: `/v3/roles` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:05:46.344032 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2404f963-672b-4142-907a-b6fac2d98182 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:05:46.345650 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2404f963-672b-4142-907a-b6fac2d98182 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:05:46.356453 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1302/2615] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 1 15:05:46 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 15:05:46.360674 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02398c00-e44b-4511-92db-a4fd3de01007 admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:05:46.371042 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02398c00-e44b-4511-92db-a4fd3de01007 admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:05:46.376938 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e8bc7c-0442-4a5a-b476-acd45b0af252 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:05:46.377580 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e8bc7c-0442-4a5a-b476-acd45b0af252 admin admin] REQUEST_METHOD: `PUT` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:05:46.377862 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e8bc7c-0442-4a5a-b476-acd45b0af252 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:05:46.378215 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e8bc7c-0442-4a5a-b476-acd45b0af252 admin admin] PATH_INFO: `/v3/projects/ff450d910bec4027a029cb0b6ddc7b2f/users/9ca1c76bc1e64ddb8fe1728b53fcb5f1/roles/5d1feef0e13d4874928bd86258c4e3b2` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:05:46.397825 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e8bc7c-0442-4a5a-b476-acd45b0af252 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff450d910bec4027a029cb0b6ddc7b2f, user_id=9ca1c76bc1e64ddb8fe1728b53fcb5f1, role_id=5d1feef0e13d4874928bd86258c4e3b2)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:05:46.400734 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e8bc7c-0442-4a5a-b476-acd45b0af252 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:05:46.416132 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1314/2616] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 1 15:05:46 2022] PUT /identity/v3/projects/ff450d910bec4027a029cb0b6ddc7b2f/users/9ca1c76bc1e64ddb8fe1728b53fcb5f1/roles/5d1feef0e13d4874928bd86258c4e3b2 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 15:05:46.427092 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad533053-d2fb-43e4-9ff2-8270a9a6afaf None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:05:46.427620 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad533053-d2fb-43e4-9ff2-8270a9a6afaf None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:05:46.427988 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad533053-d2fb-43e4-9ff2-8270a9a6afaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:05:46.458601 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-ad533053-d2fb-43e4-9ff2-8270a9a6afaf None None] MFA Rules not processed for user `9ca1c76bc1e64ddb8fe1728b53fcb5f1`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 15:05:46.460310 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-ad533053-d2fb-43e4-9ff2-8270a9a6afaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:05:46.532882 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1303/2617] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 15:05:46 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 15:05:46.537483 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e8bc7c-0442-4a5a-b476-acd45b0af252 admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:05:46.602902 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e8bc7c-0442-4a5a-b476-acd45b0af252 admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:05:46.607600 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e25d76-1396-47e6-9af2-f02cdb74e7a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:05:46.608209 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e25d76-1396-47e6-9af2-f02cdb74e7a4 admin admin] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:05:46.608492 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e25d76-1396-47e6-9af2-f02cdb74e7a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:05:46.608763 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e25d76-1396-47e6-9af2-f02cdb74e7a4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:05:46.609496 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e25d76-1396-47e6-9af2-f02cdb74e7a4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:05:46.611496 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e25d76-1396-47e6-9af2-f02cdb74e7a4 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:05:46.644162 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1315/2618] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Sep 1 15:05:46 2022] POST /identity/v3/projects => generated 367 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 15:05:46.645527 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad533053-d2fb-43e4-9ff2-8270a9a6afaf None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:05:46.660329 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad533053-d2fb-43e4-9ff2-8270a9a6afaf None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:05:46.726030 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b12e393-8ad9-4eec-a23e-7ea043e0e64e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:05:46.726790 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b12e393-8ad9-4eec-a23e-7ea043e0e64e admin admin] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:05:46.727140 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b12e393-8ad9-4eec-a23e-7ea043e0e64e admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:05:46.727487 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b12e393-8ad9-4eec-a23e-7ea043e0e64e admin admin] PATH_INFO: `/v3/users` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:05:46.728198 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b12e393-8ad9-4eec-a23e-7ea043e0e64e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:05:46.730340 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b12e393-8ad9-4eec-a23e-7ea043e0e64e admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:05:46.733875 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f30401c-6717-44ab-ac24-1fd894d82e7e None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:05:46.734417 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f30401c-6717-44ab-ac24-1fd894d82e7e None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:05:46.734897 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f30401c-6717-44ab-ac24-1fd894d82e7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:05:46.755017 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1304/2619] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Sep 1 15:05:46 2022] POST /identity/v3/users => generated 290 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 15:05:46.760265 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b12e393-8ad9-4eec-a23e-7ea043e0e64e admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:05:46.769002 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-3f30401c-6717-44ab-ac24-1fd894d82e7e None None] MFA Rules not processed for user `497118c4126444398c23cd951eed7086`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 15:05:46.771031 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-3f30401c-6717-44ab-ac24-1fd894d82e7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:05:46.772425 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b12e393-8ad9-4eec-a23e-7ea043e0e64e admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:05:46.775935 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 15:05:46.780186 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57aa33fa-ab40-4e58-9993-48a0fc615f14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:05:46.780935 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57aa33fa-ab40-4e58-9993-48a0fc615f14 admin admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:05:46.781345 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57aa33fa-ab40-4e58-9993-48a0fc615f14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:05:46.781618 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57aa33fa-ab40-4e58-9993-48a0fc615f14 admin admin] PATH_INFO: `/v3/roles` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:05:46.782354 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57aa33fa-ab40-4e58-9993-48a0fc615f14 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:05:46.783900 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57aa33fa-ab40-4e58-9993-48a0fc615f14 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:05:46.798289 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1305/2620] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 1 15:05:46 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 15:05:46.800231 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57aa33fa-ab40-4e58-9993-48a0fc615f14 admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:05:46.812916 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57aa33fa-ab40-4e58-9993-48a0fc615f14 admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:05:46.817782 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb44414-8267-48c7-b70d-28b9df52ca4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:05:46.818532 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb44414-8267-48c7-b70d-28b9df52ca4d admin admin] REQUEST_METHOD: `PUT` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:05:46.818932 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb44414-8267-48c7-b70d-28b9df52ca4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:05:46.819471 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb44414-8267-48c7-b70d-28b9df52ca4d admin admin] PATH_INFO: `/v3/projects/0fbc95f9e78644038f9f91a3b87d2484/users/1b46fcbff7b34f76a06632d929cf0440/roles/2a2adb769e55476b82e5014897e0a9c0` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:05:46.835809 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb44414-8267-48c7-b70d-28b9df52ca4d admin admin] RBAC: Authorizing `identity:create_grant(project_id=0fbc95f9e78644038f9f91a3b87d2484, user_id=1b46fcbff7b34f76a06632d929cf0440, role_id=2a2adb769e55476b82e5014897e0a9c0)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:05:46.838494 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb44414-8267-48c7-b70d-28b9df52ca4d admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:05:46.853279 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1306/2621] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 1 15:05:46 2022] PUT /identity/v3/projects/0fbc95f9e78644038f9f91a3b87d2484/users/1b46fcbff7b34f76a06632d929cf0440/roles/2a2adb769e55476b82e5014897e0a9c0 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 15:05:46.857706 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb44414-8267-48c7-b70d-28b9df52ca4d admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:05:46.859312 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1316/2622] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 15:05:46 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 15:05:46.870775 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f30401c-6717-44ab-ac24-1fd894d82e7e None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:05:46.872867 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-3f30401c-6717-44ab-ac24-1fd894d82e7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:05:46.889815 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 15:05:46.932288 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb44414-8267-48c7-b70d-28b9df52ca4d admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:05:46.938060 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792cf512-f368-484d-aa73-86d04c2bb5ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:05:46.938694 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792cf512-f368-484d-aa73-86d04c2bb5ea admin admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:05:46.938972 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792cf512-f368-484d-aa73-86d04c2bb5ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:05:46.939421 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792cf512-f368-484d-aa73-86d04c2bb5ea admin admin] PATH_INFO: `/v3/roles` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:05:46.940066 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792cf512-f368-484d-aa73-86d04c2bb5ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:05:46.941713 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792cf512-f368-484d-aa73-86d04c2bb5ea admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:05:46.944394 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f30401c-6717-44ab-ac24-1fd894d82e7e None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:05:46.951907 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1307/2623] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 1 15:05:46 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 15:05:46.955986 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792cf512-f368-484d-aa73-86d04c2bb5ea admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:05:46.966921 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792cf512-f368-484d-aa73-86d04c2bb5ea admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:05:46.971987 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ab4f19-76cc-48ae-b3eb-9f5f48af2b80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:05:46.972620 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ab4f19-76cc-48ae-b3eb-9f5f48af2b80 admin admin] REQUEST_METHOD: `PUT` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:05:46.972901 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ab4f19-76cc-48ae-b3eb-9f5f48af2b80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:05:46.973299 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ab4f19-76cc-48ae-b3eb-9f5f48af2b80 admin admin] PATH_INFO: `/v3/projects/0fbc95f9e78644038f9f91a3b87d2484/users/1b46fcbff7b34f76a06632d929cf0440/roles/2a2adb769e55476b82e5014897e0a9c0` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:05:46.975277 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ab4f19-76cc-48ae-b3eb-9f5f48af2b80 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0fbc95f9e78644038f9f91a3b87d2484, user_id=1b46fcbff7b34f76a06632d929cf0440, role_id=2a2adb769e55476b82e5014897e0a9c0)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:05:46.977916 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ab4f19-76cc-48ae-b3eb-9f5f48af2b80 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:05:46.990575 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1308/2624] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 1 15:05:46 2022] PUT /identity/v3/projects/0fbc95f9e78644038f9f91a3b87d2484/users/1b46fcbff7b34f76a06632d929cf0440/roles/2a2adb769e55476b82e5014897e0a9c0 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 15:05:46.999134 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d9f3cc-0b05-4b22-8a88-b0cb62317dbd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3b81739e34e4e648f88b6d73fcb5e46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:05:46.999736 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d9f3cc-0b05-4b22-8a88-b0cb62317dbd service cinder] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:05:47.000016 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d9f3cc-0b05-4b22-8a88-b0cb62317dbd service cinder] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:05:47.000286 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d9f3cc-0b05-4b22-8a88-b0cb62317dbd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:05:47.002119 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-14d9f3cc-0b05-4b22-8a88-b0cb62317dbd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:05:47.002448 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bac00b7-9a9b-4e67-bb98-623eaf18793a None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:05:47.002736 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bac00b7-9a9b-4e67-bb98-623eaf18793a None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:05:47.003006 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bac00b7-9a9b-4e67-bb98-623eaf18793a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:05:47.029930 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-3bac00b7-9a9b-4e67-bb98-623eaf18793a None None] MFA Rules not processed for user `1b46fcbff7b34f76a06632d929cf0440`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 15:05:47.031166 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-3bac00b7-9a9b-4e67-bb98-623eaf18793a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:05:47.063645 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d9f3cc-0b05-4b22-8a88-b0cb62317dbd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:05:47.065099 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d9f3cc-0b05-4b22-8a88-b0cb62317dbd service cinder] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:05:47.075159 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1317/2625] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 15:05:46 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 15:05:47.101494 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1309/2626] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 15:05:47 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 15:05:47.112580 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d9f3cc-0b05-4b22-8a88-b0cb62317dbd service cinder] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:05:47.114114 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-14d9f3cc-0b05-4b22-8a88-b0cb62317dbd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:05:47.126423 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d9f3cc-0b05-4b22-8a88-b0cb62317dbd service cinder] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:05:47.136344 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 15:05:47.196692 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3296d4-97c1-4a94-bee2-4f3e57caaa81 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3b81739e34e4e648f88b6d73fcb5e46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:05:47.197298 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3296d4-97c1-4a94-bee2-4f3e57caaa81 service cinder] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:05:47.197576 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3296d4-97c1-4a94-bee2-4f3e57caaa81 service cinder] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:05:47.198560 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3296d4-97c1-4a94-bee2-4f3e57caaa81 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:05:47.210800 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3296d4-97c1-4a94-bee2-4f3e57caaa81 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:05:47.212419 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3296d4-97c1-4a94-bee2-4f3e57caaa81 service cinder] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:05:47.224893 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1318/2627] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 15:05:47 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 15:05:48.536348 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bac00b7-9a9b-4e67-bb98-623eaf18793a None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:05:48.547513 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bac00b7-9a9b-4e67-bb98-623eaf18793a None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:05:48.551391 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c8c840-fb16-4f96-9c9e-6a44fedc5d52 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3b81739e34e4e648f88b6d73fcb5e46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:05:48.551997 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c8c840-fb16-4f96-9c9e-6a44fedc5d52 service cinder] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:05:48.552266 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c8c840-fb16-4f96-9c9e-6a44fedc5d52 service cinder] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:05:48.552530 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c8c840-fb16-4f96-9c9e-6a44fedc5d52 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:05:48.624338 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c8c840-fb16-4f96-9c9e-6a44fedc5d52 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:05:48.625781 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c8c840-fb16-4f96-9c9e-6a44fedc5d52 service cinder] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:05:48.636835 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1310/2628] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 15:05:48 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 15:05:48.696339 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3296d4-97c1-4a94-bee2-4f3e57caaa81 service cinder] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:05:48.768404 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3296d4-97c1-4a94-bee2-4f3e57caaa81 service cinder] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:05:48.773188 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a74331-015a-4f5a-943f-a184a95961d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:05:48.773836 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a74331-015a-4f5a-943f-a184a95961d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:05:48.774093 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a74331-015a-4f5a-943f-a184a95961d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:05:48.774393 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a74331-015a-4f5a-943f-a184a95961d4 admin admin] PATH_INFO: `/v3/users/9ca1c76bc1e64ddb8fe1728b53fcb5f1` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:05:48.775522 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a74331-015a-4f5a-943f-a184a95961d4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9ca1c76bc1e64ddb8fe1728b53fcb5f1)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:05:48.777969 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a74331-015a-4f5a-943f-a184a95961d4 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:05:48.827117 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-b5a74331-015a-4f5a-943f-a184a95961d4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9ca1c76bc1e64ddb8fe1728b53fcb5f1'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:05:48.834247 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-b5a74331-015a-4f5a-943f-a184a95961d4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9ca1c76bc1e64ddb8fe1728b53fcb5f1'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:05:48.834898 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-b5a74331-015a-4f5a-943f-a184a95961d4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9ca1c76bc1e64ddb8fe1728b53fcb5f1'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:05:48.849165 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-b5a74331-015a-4f5a-943f-a184a95961d4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9ca1c76bc1e64ddb8fe1728b53fcb5f1'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:05:48.849864 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-b5a74331-015a-4f5a-943f-a184a95961d4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9ca1c76bc1e64ddb8fe1728b53fcb5f1'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:05:48.863117 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1319/2629] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Sep 1 15:05:48 2022] DELETE /identity/v3/users/9ca1c76bc1e64ddb8fe1728b53fcb5f1 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 15:05:48.867519 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a6e70e-f68c-4d1d-a702-9f06c4faae1c None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:05:48.867812 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a6e70e-f68c-4d1d-a702-9f06c4faae1c None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:05:48.868083 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a6e70e-f68c-4d1d-a702-9f06c4faae1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:05:48.887354 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-e7a6e70e-f68c-4d1d-a702-9f06c4faae1c None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 15:05:48.888630 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-e7a6e70e-f68c-4d1d-a702-9f06c4faae1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:05:48.909563 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 15:05:48.948449 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1311/2630] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 15:05:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 15:05:48.958274 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a74331-015a-4f5a-943f-a184a95961d4 admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:05:48.959704 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-b5a74331-015a-4f5a-943f-a184a95961d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:05:48.995271 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 15:05:49.020926 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a74331-015a-4f5a-943f-a184a95961d4 admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:05:49.024593 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391b7f0e-c01e-4291-a7de-8992eae46183 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2790149639aa418bbc4e4982e93eda44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:05:49.025191 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391b7f0e-c01e-4291-a7de-8992eae46183 service neutron] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:05:49.025472 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391b7f0e-c01e-4291-a7de-8992eae46183 service neutron] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:05:49.025740 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391b7f0e-c01e-4291-a7de-8992eae46183 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:05:49.035323 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391b7f0e-c01e-4291-a7de-8992eae46183 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:05:49.036810 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391b7f0e-c01e-4291-a7de-8992eae46183 service neutron] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:05:49.047675 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1320/2631] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 15:05:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 15:05:49.312339 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a6e70e-f68c-4d1d-a702-9f06c4faae1c None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:05:49.313845 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-e7a6e70e-f68c-4d1d-a702-9f06c4faae1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:05:49.328245 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a6e70e-f68c-4d1d-a702-9f06c4faae1c None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:05:49.333070 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070ce5a9-0cc5-4543-bb4c-079854571ced admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:05:49.333695 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070ce5a9-0cc5-4543-bb4c-079854571ced admin admin] REQUEST_METHOD: `DELETE` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:05:49.334052 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070ce5a9-0cc5-4543-bb4c-079854571ced admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:05:49.334357 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070ce5a9-0cc5-4543-bb4c-079854571ced admin admin] PATH_INFO: `/v3/projects/ff450d910bec4027a029cb0b6ddc7b2f` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:05:49.335467 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070ce5a9-0cc5-4543-bb4c-079854571ced admin admin] RBAC: Authorizing `identity:delete_project(project_id=ff450d910bec4027a029cb0b6ddc7b2f)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:05:49.337388 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070ce5a9-0cc5-4543-bb4c-079854571ced admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:05:49.382767 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-070ce5a9-0cc5-4543-bb4c-079854571ced admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ff450d910bec4027a029cb0b6ddc7b2f', 'request_id': 'req-070ce5a9-0cc5-4543-bb4c-079854571ced'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:05:49.389374 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-070ce5a9-0cc5-4543-bb4c-079854571ced admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ff450d910bec4027a029cb0b6ddc7b2f', 'request_id': 'req-070ce5a9-0cc5-4543-bb4c-079854571ced'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:05:49.396661 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-070ce5a9-0cc5-4543-bb4c-079854571ced admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ff450d910bec4027a029cb0b6ddc7b2f', 'request_id': 'req-070ce5a9-0cc5-4543-bb4c-079854571ced'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:05:49.407647 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-070ce5a9-0cc5-4543-bb4c-079854571ced admin admin] The token cache is being invalidate because project ff450d910bec4027a029cb0b6ddc7b2f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81341) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 15:05:49.407930 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-070ce5a9-0cc5-4543-bb4c-079854571ced admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:05:49.408544 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-070ce5a9-0cc5-4543-bb4c-079854571ced admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:05:49.409997 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1312/2632] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Sep 1 15:05:49 2022] DELETE /identity/v3/projects/ff450d910bec4027a029cb0b6ddc7b2f => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 15:05:49.414049 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391b7f0e-c01e-4291-a7de-8992eae46183 service neutron] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:05:49.423993 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391b7f0e-c01e-4291-a7de-8992eae46183 service neutron] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:05:49.481064 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9e077c-be74-4b31-9a91-80eaed22afcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:05:49.481591 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9e077c-be74-4b31-9a91-80eaed22afcb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:05:49.481861 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9e077c-be74-4b31-9a91-80eaed22afcb admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:05:49.482125 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9e077c-be74-4b31-9a91-80eaed22afcb admin admin] PATH_INFO: `/v3/users/1b46fcbff7b34f76a06632d929cf0440` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:05:49.483266 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9e077c-be74-4b31-9a91-80eaed22afcb admin admin] RBAC: Authorizing `identity:delete_user(user_id=1b46fcbff7b34f76a06632d929cf0440)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:05:49.485172 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9e077c-be74-4b31-9a91-80eaed22afcb admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:05:49.533740 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-1f9e077c-be74-4b31-9a91-80eaed22afcb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1b46fcbff7b34f76a06632d929cf0440'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:05:49.540854 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-1f9e077c-be74-4b31-9a91-80eaed22afcb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1b46fcbff7b34f76a06632d929cf0440'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:05:49.541612 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-1f9e077c-be74-4b31-9a91-80eaed22afcb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1b46fcbff7b34f76a06632d929cf0440'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:05:49.553703 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-1f9e077c-be74-4b31-9a91-80eaed22afcb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1b46fcbff7b34f76a06632d929cf0440'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:05:49.554351 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-1f9e077c-be74-4b31-9a91-80eaed22afcb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1b46fcbff7b34f76a06632d929cf0440'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:05:49.566976 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1321/2633] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Sep 1 15:05:49 2022] DELETE /identity/v3/users/1b46fcbff7b34f76a06632d929cf0440 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 15:05:49.843400 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070ce5a9-0cc5-4543-bb4c-079854571ced admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:05:49.844955 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-070ce5a9-0cc5-4543-bb4c-079854571ced admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:05:49.863078 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070ce5a9-0cc5-4543-bb4c-079854571ced admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:05:49.947089 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c59ebd-b241-4934-9565-ca9d180f55a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:05:49.947910 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c59ebd-b241-4934-9565-ca9d180f55a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:05:49.948300 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c59ebd-b241-4934-9565-ca9d180f55a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:05:49.948729 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c59ebd-b241-4934-9565-ca9d180f55a5 admin admin] PATH_INFO: `/v3/projects/0fbc95f9e78644038f9f91a3b87d2484` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:05:49.950573 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c59ebd-b241-4934-9565-ca9d180f55a5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0fbc95f9e78644038f9f91a3b87d2484)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:05:49.952583 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c59ebd-b241-4934-9565-ca9d180f55a5 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:05:50.013968 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-e7c59ebd-b241-4934-9565-ca9d180f55a5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0fbc95f9e78644038f9f91a3b87d2484', 'request_id': 'req-e7c59ebd-b241-4934-9565-ca9d180f55a5'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:05:50.017355 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 15:05:50.022733 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-e7c59ebd-b241-4934-9565-ca9d180f55a5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0fbc95f9e78644038f9f91a3b87d2484', 'request_id': 'req-e7c59ebd-b241-4934-9565-ca9d180f55a5'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:05:50.034541 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-e7c59ebd-b241-4934-9565-ca9d180f55a5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0fbc95f9e78644038f9f91a3b87d2484', 'request_id': 'req-e7c59ebd-b241-4934-9565-ca9d180f55a5'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:05:50.048252 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-e7c59ebd-b241-4934-9565-ca9d180f55a5 admin admin] The token cache is being invalidate because project 0fbc95f9e78644038f9f91a3b87d2484 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81341) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 15:05:50.048751 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-e7c59ebd-b241-4934-9565-ca9d180f55a5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:05:50.049522 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-e7c59ebd-b241-4934-9565-ca9d180f55a5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:05:50.054275 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1313/2634] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Sep 1 15:05:49 2022] DELETE /identity/v3/projects/0fbc95f9e78644038f9f91a3b87d2484 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 15:05:50.056690 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f242f7d-0b22-45ea-93ad-740a6d7d66eb None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:05:50.057145 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f242f7d-0b22-45ea-93ad-740a6d7d66eb None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:05:50.057230 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f242f7d-0b22-45ea-93ad-740a6d7d66eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:05:50.078305 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-6f242f7d-0b22-45ea-93ad-740a6d7d66eb None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 15:05:50.079576 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-6f242f7d-0b22-45ea-93ad-740a6d7d66eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:05:50.141957 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 15:05:50.160519 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1322/2635] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 15:05:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 15:05:50.169233 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7050e2d-43e6-4ba8-955d-daa1d07869ea None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:05:50.169537 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7050e2d-43e6-4ba8-955d-daa1d07869ea None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:05:50.169812 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7050e2d-43e6-4ba8-955d-daa1d07869ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:05:50.191675 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-d7050e2d-43e6-4ba8-955d-daa1d07869ea None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 15:05:50.197584 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-d7050e2d-43e6-4ba8-955d-daa1d07869ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:05:50.231656 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1314/2636] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 15:05:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 15:05:50.236984 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f242f7d-0b22-45ea-93ad-740a6d7d66eb None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:05:50.257329 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f242f7d-0b22-45ea-93ad-740a6d7d66eb None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:05:50.262415 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d998ae-2194-4014-946f-8dba83f67869 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:05:50.262978 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d998ae-2194-4014-946f-8dba83f67869 admin admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:05:50.263275 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d998ae-2194-4014-946f-8dba83f67869 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:05:50.263508 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d998ae-2194-4014-946f-8dba83f67869 admin admin] PATH_INFO: `/v3/domains` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:05:50.264156 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d998ae-2194-4014-946f-8dba83f67869 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 15:05:50.264455 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d998ae-2194-4014-946f-8dba83f67869 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:05:50.266043 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d998ae-2194-4014-946f-8dba83f67869 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:05:50.281007 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1323/2637] 10.222.0.33 () {58 vars in 1153 bytes} [Thu Sep 1 15:05:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 15:05:50.285418 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7050e2d-43e6-4ba8-955d-daa1d07869ea None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:05:50.297296 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7050e2d-43e6-4ba8-955d-daa1d07869ea None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:05:50.301993 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffafa0b-fe5d-4697-8015-3cfc89ed14c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:05:50.302665 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffafa0b-fe5d-4697-8015-3cfc89ed14c3 admin admin] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:05:50.302945 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffafa0b-fe5d-4697-8015-3cfc89ed14c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:05:50.303215 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffafa0b-fe5d-4697-8015-3cfc89ed14c3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:05:50.303905 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffafa0b-fe5d-4697-8015-3cfc89ed14c3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:05:50.305989 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffafa0b-fe5d-4697-8015-3cfc89ed14c3 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:05:50.332290 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1315/2638] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Sep 1 15:05:50 2022] POST /identity/v3/projects => generated 391 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 15:05:50.336465 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d998ae-2194-4014-946f-8dba83f67869 admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:05:50.346766 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d998ae-2194-4014-946f-8dba83f67869 admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:05:50.402432 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39767618-f7df-4b9d-a00e-2eadb7e00ec1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:05:50.403038 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39767618-f7df-4b9d-a00e-2eadb7e00ec1 admin admin] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:05:50.403315 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39767618-f7df-4b9d-a00e-2eadb7e00ec1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:05:50.403584 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39767618-f7df-4b9d-a00e-2eadb7e00ec1 admin admin] PATH_INFO: `/v3/users` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:05:50.404217 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39767618-f7df-4b9d-a00e-2eadb7e00ec1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:05:50.406167 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39767618-f7df-4b9d-a00e-2eadb7e00ec1 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:05:50.427801 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1324/2639] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Sep 1 15:05:50 2022] POST /identity/v3/users => generated 296 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 15:05:50.431930 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffafa0b-fe5d-4697-8015-3cfc89ed14c3 admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:05:50.442611 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffafa0b-fe5d-4697-8015-3cfc89ed14c3 admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:05:50.447712 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5faed1-f02d-427c-8e9c-617140c87156 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:05:50.448321 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5faed1-f02d-427c-8e9c-617140c87156 admin admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:05:50.448601 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5faed1-f02d-427c-8e9c-617140c87156 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:05:50.448988 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5faed1-f02d-427c-8e9c-617140c87156 admin admin] PATH_INFO: `/v3/roles` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:05:50.449639 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5faed1-f02d-427c-8e9c-617140c87156 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:05:50.451676 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5faed1-f02d-427c-8e9c-617140c87156 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:05:50.462863 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1316/2640] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 1 15:05:50 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 15:05:50.466733 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39767618-f7df-4b9d-a00e-2eadb7e00ec1 admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:05:50.476536 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39767618-f7df-4b9d-a00e-2eadb7e00ec1 admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:05:50.481863 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b8b926-d86e-4010-8e41-03dd1aa41b85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:05:50.482941 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b8b926-d86e-4010-8e41-03dd1aa41b85 admin admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:05:50.482941 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b8b926-d86e-4010-8e41-03dd1aa41b85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:05:50.483166 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b8b926-d86e-4010-8e41-03dd1aa41b85 admin admin] PATH_INFO: `/v3/roles` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:05:50.483782 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b8b926-d86e-4010-8e41-03dd1aa41b85 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:05:50.485311 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b8b926-d86e-4010-8e41-03dd1aa41b85 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:05:50.497636 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1325/2641] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 1 15:05:50 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 15:05:50.502003 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5faed1-f02d-427c-8e9c-617140c87156 admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:05:50.513090 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5faed1-f02d-427c-8e9c-617140c87156 admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:05:50.518532 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a308a16c-2044-474f-8823-28fd39de2d99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:05:50.519234 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a308a16c-2044-474f-8823-28fd39de2d99 admin admin] REQUEST_METHOD: `PUT` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:05:50.519511 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a308a16c-2044-474f-8823-28fd39de2d99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:05:50.519783 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a308a16c-2044-474f-8823-28fd39de2d99 admin admin] PATH_INFO: `/v3/projects/17193a880944482385a0a74c49a6e54b/users/90986d1a465c40298661876a30444005/roles/5d1feef0e13d4874928bd86258c4e3b2` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:05:50.535302 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a308a16c-2044-474f-8823-28fd39de2d99 admin admin] RBAC: Authorizing `identity:create_grant(project_id=17193a880944482385a0a74c49a6e54b, user_id=90986d1a465c40298661876a30444005, role_id=5d1feef0e13d4874928bd86258c4e3b2)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:05:50.537880 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a308a16c-2044-474f-8823-28fd39de2d99 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:05:50.552063 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1317/2642] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 1 15:05:50 2022] PUT /identity/v3/projects/17193a880944482385a0a74c49a6e54b/users/90986d1a465c40298661876a30444005/roles/5d1feef0e13d4874928bd86258c4e3b2 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 15:05:50.562985 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2c3386-31f9-4791-9d31-455e415da3ae None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:05:50.563391 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2c3386-31f9-4791-9d31-455e415da3ae None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:05:50.563753 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2c3386-31f9-4791-9d31-455e415da3ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:05:50.593964 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-ea2c3386-31f9-4791-9d31-455e415da3ae None None] MFA Rules not processed for user `90986d1a465c40298661876a30444005`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 15:05:50.595283 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-ea2c3386-31f9-4791-9d31-455e415da3ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:05:50.661614 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1326/2643] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 15:05:50 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 15:05:50.666396 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a308a16c-2044-474f-8823-28fd39de2d99 admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:05:50.728703 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a308a16c-2044-474f-8823-28fd39de2d99 admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:05:50.733427 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc39e1e-9416-4dec-9228-5076434a7114 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:05:50.734043 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc39e1e-9416-4dec-9228-5076434a7114 admin admin] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:05:50.734418 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc39e1e-9416-4dec-9228-5076434a7114 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:05:50.734696 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc39e1e-9416-4dec-9228-5076434a7114 admin admin] PATH_INFO: `/v3/projects` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:05:50.735319 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc39e1e-9416-4dec-9228-5076434a7114 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:05:50.737249 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc39e1e-9416-4dec-9228-5076434a7114 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:05:50.764025 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1318/2644] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Sep 1 15:05:50 2022] POST /identity/v3/projects => generated 393 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 15:05:50.768076 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2c3386-31f9-4791-9d31-455e415da3ae None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:05:50.779159 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2c3386-31f9-4791-9d31-455e415da3ae None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:05:50.824812 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 15:05:50.835216 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f16d09-0449-4382-afac-e20b4d517aa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:05:50.835813 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f16d09-0449-4382-afac-e20b4d517aa6 admin admin] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:05:50.836087 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f16d09-0449-4382-afac-e20b4d517aa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:05:50.836354 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f16d09-0449-4382-afac-e20b4d517aa6 admin admin] PATH_INFO: `/v3/users` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:05:50.836978 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f16d09-0449-4382-afac-e20b4d517aa6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:05:50.839541 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f16d09-0449-4382-afac-e20b4d517aa6 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:05:50.859638 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1327/2645] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Sep 1 15:05:50 2022] POST /identity/v3/users => generated 303 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 15:05:50.863674 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc39e1e-9416-4dec-9228-5076434a7114 admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:05:50.873875 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc39e1e-9416-4dec-9228-5076434a7114 admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:05:50.878476 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbeec6d-3ec4-4f49-9bce-59360477f7ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:05:50.879065 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbeec6d-3ec4-4f49-9bce-59360477f7ec admin admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:05:50.879345 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbeec6d-3ec4-4f49-9bce-59360477f7ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:05:50.879616 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbeec6d-3ec4-4f49-9bce-59360477f7ec admin admin] PATH_INFO: `/v3/roles` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:05:50.880254 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbeec6d-3ec4-4f49-9bce-59360477f7ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:05:50.881869 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbeec6d-3ec4-4f49-9bce-59360477f7ec admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:05:50.892926 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1319/2646] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 1 15:05:50 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 15:05:50.896773 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f16d09-0449-4382-afac-e20b4d517aa6 admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:05:50.907169 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f16d09-0449-4382-afac-e20b4d517aa6 admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:05:50.911676 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84cb876-7ff9-47a2-bf2c-71aa357ce687 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:05:50.912304 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84cb876-7ff9-47a2-bf2c-71aa357ce687 admin admin] REQUEST_METHOD: `PUT` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:05:50.912579 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84cb876-7ff9-47a2-bf2c-71aa357ce687 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:05:50.912924 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84cb876-7ff9-47a2-bf2c-71aa357ce687 admin admin] PATH_INFO: `/v3/projects/bc113c0260f14bdb9a2d5ddd7eae08e3/users/fef33572d4ee4fde878f88d4eecc5d87/roles/2a2adb769e55476b82e5014897e0a9c0` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:05:50.927671 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84cb876-7ff9-47a2-bf2c-71aa357ce687 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc113c0260f14bdb9a2d5ddd7eae08e3, user_id=fef33572d4ee4fde878f88d4eecc5d87, role_id=2a2adb769e55476b82e5014897e0a9c0)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:05:50.930373 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84cb876-7ff9-47a2-bf2c-71aa357ce687 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:05:50.943473 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1328/2647] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 1 15:05:50 2022] PUT /identity/v3/projects/bc113c0260f14bdb9a2d5ddd7eae08e3/users/fef33572d4ee4fde878f88d4eecc5d87/roles/2a2adb769e55476b82e5014897e0a9c0 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 15:05:50.947422 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbeec6d-3ec4-4f49-9bce-59360477f7ec admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:05:50.958177 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbeec6d-3ec4-4f49-9bce-59360477f7ec admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:05:51.004720 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 15:05:51.015853 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef673939-46c7-4919-8618-635a2c62f3a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:05:51.016445 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef673939-46c7-4919-8618-635a2c62f3a7 admin admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:05:51.016720 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef673939-46c7-4919-8618-635a2c62f3a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:05:51.016985 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef673939-46c7-4919-8618-635a2c62f3a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:05:51.017609 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef673939-46c7-4919-8618-635a2c62f3a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:05:51.019237 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef673939-46c7-4919-8618-635a2c62f3a7 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:05:51.028662 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1320/2648] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 1 15:05:50 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 15:05:51.033100 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84cb876-7ff9-47a2-bf2c-71aa357ce687 admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:05:51.043926 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84cb876-7ff9-47a2-bf2c-71aa357ce687 admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:05:51.049187 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a8d968-ae7a-4c0e-969b-6429950b9853 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:05:51.049837 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a8d968-ae7a-4c0e-969b-6429950b9853 admin admin] REQUEST_METHOD: `PUT` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:05:51.050112 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a8d968-ae7a-4c0e-969b-6429950b9853 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:05:51.050452 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a8d968-ae7a-4c0e-969b-6429950b9853 admin admin] PATH_INFO: `/v3/projects/bc113c0260f14bdb9a2d5ddd7eae08e3/users/fef33572d4ee4fde878f88d4eecc5d87/roles/2a2adb769e55476b82e5014897e0a9c0` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:05:51.052483 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a8d968-ae7a-4c0e-969b-6429950b9853 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc113c0260f14bdb9a2d5ddd7eae08e3, user_id=fef33572d4ee4fde878f88d4eecc5d87, role_id=2a2adb769e55476b82e5014897e0a9c0)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:05:51.055138 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a8d968-ae7a-4c0e-969b-6429950b9853 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:05:51.066019 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1329/2649] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 1 15:05:51 2022] PUT /identity/v3/projects/bc113c0260f14bdb9a2d5ddd7eae08e3/users/fef33572d4ee4fde878f88d4eecc5d87/roles/2a2adb769e55476b82e5014897e0a9c0 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 15:05:51.077918 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03824729-c176-42f6-948b-b405f1e07588 None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:05:51.078152 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03824729-c176-42f6-948b-b405f1e07588 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:05:51.078436 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03824729-c176-42f6-948b-b405f1e07588 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:05:51.105609 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-03824729-c176-42f6-948b-b405f1e07588 None None] MFA Rules not processed for user `fef33572d4ee4fde878f88d4eecc5d87`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 15:05:51.106920 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-03824729-c176-42f6-948b-b405f1e07588 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:05:51.179193 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1321/2650] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 15:05:51 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 15:05:51.190763 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a8d968-ae7a-4c0e-969b-6429950b9853 admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:05:51.192490 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-20a8d968-ae7a-4c0e-969b-6429950b9853 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:05:51.250377 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a8d968-ae7a-4c0e-969b-6429950b9853 admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:05:51.253989 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8ca908-7bcb-43d2-98bb-69e510ac69e4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3b81739e34e4e648f88b6d73fcb5e46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:05:51.254610 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8ca908-7bcb-43d2-98bb-69e510ac69e4 service cinder] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:05:51.254885 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8ca908-7bcb-43d2-98bb-69e510ac69e4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:05:51.255150 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8ca908-7bcb-43d2-98bb-69e510ac69e4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:05:51.264435 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8ca908-7bcb-43d2-98bb-69e510ac69e4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:05:51.265855 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8ca908-7bcb-43d2-98bb-69e510ac69e4 service cinder] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:05:51.275252 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1330/2651] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 15:05:51 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 15:05:51.327363 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03824729-c176-42f6-948b-b405f1e07588 None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:05:51.328821 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-03824729-c176-42f6-948b-b405f1e07588 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:05:51.340820 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03824729-c176-42f6-948b-b405f1e07588 None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:05:51.344475 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b851dc4-ed32-43a8-95ee-6c6f12d524b6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3b81739e34e4e648f88b6d73fcb5e46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:05:51.345072 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b851dc4-ed32-43a8-95ee-6c6f12d524b6 service cinder] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:05:51.345351 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b851dc4-ed32-43a8-95ee-6c6f12d524b6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:05:51.345657 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b851dc4-ed32-43a8-95ee-6c6f12d524b6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:05:51.403695 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b851dc4-ed32-43a8-95ee-6c6f12d524b6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:05:51.405181 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b851dc4-ed32-43a8-95ee-6c6f12d524b6 service cinder] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:05:51.414941 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1322/2652] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 15:05:51 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 15:05:54.767269 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8ca908-7bcb-43d2-98bb-69e510ac69e4 service cinder] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:05:54.830466 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8ca908-7bcb-43d2-98bb-69e510ac69e4 service cinder] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:05:54.834988 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9401a334-71a4-42ba-9568-402014e046ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:05:54.835610 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9401a334-71a4-42ba-9568-402014e046ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:05:54.835885 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9401a334-71a4-42ba-9568-402014e046ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:05:54.836152 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9401a334-71a4-42ba-9568-402014e046ab admin admin] PATH_INFO: `/v3/users/90986d1a465c40298661876a30444005` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:05:54.837435 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9401a334-71a4-42ba-9568-402014e046ab admin admin] RBAC: Authorizing `identity:delete_user(user_id=90986d1a465c40298661876a30444005)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:05:54.839450 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9401a334-71a4-42ba-9568-402014e046ab admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:05:54.888891 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-9401a334-71a4-42ba-9568-402014e046ab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '90986d1a465c40298661876a30444005'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:05:54.890575 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 15:05:54.896864 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-9401a334-71a4-42ba-9568-402014e046ab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '90986d1a465c40298661876a30444005'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:05:54.898500 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-9401a334-71a4-42ba-9568-402014e046ab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '90986d1a465c40298661876a30444005'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:05:54.913262 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-9401a334-71a4-42ba-9568-402014e046ab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '90986d1a465c40298661876a30444005'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:05:54.913971 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-9401a334-71a4-42ba-9568-402014e046ab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '90986d1a465c40298661876a30444005'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:05:54.926724 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1331/2653] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Sep 1 15:05:54 2022] DELETE /identity/v3/users/90986d1a465c40298661876a30444005 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 15:05:54.931177 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d16034a-c643-46be-8752-625d4571f18c None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:05:54.931679 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d16034a-c643-46be-8752-625d4571f18c None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:05:54.931777 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d16034a-c643-46be-8752-625d4571f18c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:05:54.951502 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-6d16034a-c643-46be-8752-625d4571f18c None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 15:05:54.952618 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-6d16034a-c643-46be-8752-625d4571f18c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:05:55.014739 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1323/2654] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 15:05:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 15:05:55.024110 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9401a334-71a4-42ba-9568-402014e046ab admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:05:55.025674 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-9401a334-71a4-42ba-9568-402014e046ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:05:55.087244 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9401a334-71a4-42ba-9568-402014e046ab admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:05:55.090901 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b62b248-8abc-453f-82dd-f0deff9807b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2790149639aa418bbc4e4982e93eda44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:05:55.091504 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b62b248-8abc-453f-82dd-f0deff9807b3 service neutron] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:05:55.091782 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b62b248-8abc-453f-82dd-f0deff9807b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:05:55.092051 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b62b248-8abc-453f-82dd-f0deff9807b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:05:55.100960 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b62b248-8abc-453f-82dd-f0deff9807b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:05:55.102416 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b62b248-8abc-453f-82dd-f0deff9807b3 service neutron] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:05:55.114414 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1332/2655] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 15:05:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 15:05:55.447975 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d16034a-c643-46be-8752-625d4571f18c None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:05:55.449975 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-6d16034a-c643-46be-8752-625d4571f18c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:05:55.464174 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d16034a-c643-46be-8752-625d4571f18c None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:05:55.470069 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cea224a-78af-49a4-a43a-f87676d2eab0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:05:55.471370 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cea224a-78af-49a4-a43a-f87676d2eab0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:05:55.471370 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cea224a-78af-49a4-a43a-f87676d2eab0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:05:55.471583 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cea224a-78af-49a4-a43a-f87676d2eab0 admin admin] PATH_INFO: `/v3/projects/17193a880944482385a0a74c49a6e54b` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:05:55.473072 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cea224a-78af-49a4-a43a-f87676d2eab0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=17193a880944482385a0a74c49a6e54b)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:05:55.475434 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cea224a-78af-49a4-a43a-f87676d2eab0 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:05:55.483466 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 15:05:55.535032 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-8cea224a-78af-49a4-a43a-f87676d2eab0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '17193a880944482385a0a74c49a6e54b', 'request_id': 'req-8cea224a-78af-49a4-a43a-f87676d2eab0'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:05:55.541232 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-8cea224a-78af-49a4-a43a-f87676d2eab0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '17193a880944482385a0a74c49a6e54b', 'request_id': 'req-8cea224a-78af-49a4-a43a-f87676d2eab0'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:05:55.550110 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-8cea224a-78af-49a4-a43a-f87676d2eab0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '17193a880944482385a0a74c49a6e54b', 'request_id': 'req-8cea224a-78af-49a4-a43a-f87676d2eab0'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:05:55.562090 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-8cea224a-78af-49a4-a43a-f87676d2eab0 admin admin] The token cache is being invalidate because project 17193a880944482385a0a74c49a6e54b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81341) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 15:05:55.562434 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-8cea224a-78af-49a4-a43a-f87676d2eab0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:05:55.563109 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-8cea224a-78af-49a4-a43a-f87676d2eab0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:05:55.564735 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1324/2656] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Sep 1 15:05:55 2022] DELETE /identity/v3/projects/17193a880944482385a0a74c49a6e54b => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 15:05:55.569014 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b62b248-8abc-453f-82dd-f0deff9807b3 service neutron] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:05:55.578843 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b62b248-8abc-453f-82dd-f0deff9807b3 service neutron] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:05:55.643745 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf946618-1bbf-4d9e-9636-0cdea5013813 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:05:55.644373 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf946618-1bbf-4d9e-9636-0cdea5013813 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:05:55.644698 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf946618-1bbf-4d9e-9636-0cdea5013813 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:05:55.644969 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf946618-1bbf-4d9e-9636-0cdea5013813 admin admin] PATH_INFO: `/v3/users/fef33572d4ee4fde878f88d4eecc5d87` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:05:55.646109 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf946618-1bbf-4d9e-9636-0cdea5013813 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fef33572d4ee4fde878f88d4eecc5d87)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:05:55.648457 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf946618-1bbf-4d9e-9636-0cdea5013813 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:05:55.714680 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-cf946618-1bbf-4d9e-9636-0cdea5013813 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fef33572d4ee4fde878f88d4eecc5d87'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:05:55.722631 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-cf946618-1bbf-4d9e-9636-0cdea5013813 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fef33572d4ee4fde878f88d4eecc5d87'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:05:55.723238 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-cf946618-1bbf-4d9e-9636-0cdea5013813 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fef33572d4ee4fde878f88d4eecc5d87'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:05:55.738988 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:INSERT=2,keystone:DELETE=13,keystone:SELECT=85 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 15:05:55.740329 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-cf946618-1bbf-4d9e-9636-0cdea5013813 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fef33572d4ee4fde878f88d4eecc5d87'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:05:55.741908 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-cf946618-1bbf-4d9e-9636-0cdea5013813 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fef33572d4ee4fde878f88d4eecc5d87'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:05:55.757491 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1333/2657] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Sep 1 15:05:55 2022] DELETE /identity/v3/users/fef33572d4ee4fde878f88d4eecc5d87 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 15:05:56.201839 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cea224a-78af-49a4-a43a-f87676d2eab0 admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:05:56.203870 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-8cea224a-78af-49a4-a43a-f87676d2eab0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:05:56.221453 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cea224a-78af-49a4-a43a-f87676d2eab0 admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:05:56.291369 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7394d5-e9b3-4643-b0ea-60029ae3c08b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:05:56.292031 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7394d5-e9b3-4643-b0ea-60029ae3c08b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:05:56.292371 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7394d5-e9b3-4643-b0ea-60029ae3c08b admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:05:56.292722 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7394d5-e9b3-4643-b0ea-60029ae3c08b admin admin] PATH_INFO: `/v3/projects/bc113c0260f14bdb9a2d5ddd7eae08e3` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:05:56.293929 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7394d5-e9b3-4643-b0ea-60029ae3c08b admin admin] RBAC: Authorizing `identity:delete_project(project_id=bc113c0260f14bdb9a2d5ddd7eae08e3)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:05:56.296296 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7394d5-e9b3-4643-b0ea-60029ae3c08b admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:05:56.353195 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-bc7394d5-e9b3-4643-b0ea-60029ae3c08b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bc113c0260f14bdb9a2d5ddd7eae08e3', 'request_id': 'req-bc7394d5-e9b3-4643-b0ea-60029ae3c08b'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:05:56.360340 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-bc7394d5-e9b3-4643-b0ea-60029ae3c08b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bc113c0260f14bdb9a2d5ddd7eae08e3', 'request_id': 'req-bc7394d5-e9b3-4643-b0ea-60029ae3c08b'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:05:56.370531 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-bc7394d5-e9b3-4643-b0ea-60029ae3c08b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bc113c0260f14bdb9a2d5ddd7eae08e3', 'request_id': 'req-bc7394d5-e9b3-4643-b0ea-60029ae3c08b'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:05:56.383093 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-bc7394d5-e9b3-4643-b0ea-60029ae3c08b admin admin] The token cache is being invalidate because project bc113c0260f14bdb9a2d5ddd7eae08e3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81341) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 15:05:56.383390 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-bc7394d5-e9b3-4643-b0ea-60029ae3c08b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:05:56.384052 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-bc7394d5-e9b3-4643-b0ea-60029ae3c08b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:05:56.386491 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1325/2658] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Sep 1 15:05:56 2022] DELETE /identity/v3/projects/bc113c0260f14bdb9a2d5ddd7eae08e3 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 15:05:56.391206 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acec56ad-a747-4601-a005-3e1fb6beac61 None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:05:56.391707 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acec56ad-a747-4601-a005-3e1fb6beac61 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:05:56.391802 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acec56ad-a747-4601-a005-3e1fb6beac61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:05:56.412304 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-acec56ad-a747-4601-a005-3e1fb6beac61 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 15:05:56.413623 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-acec56ad-a747-4601-a005-3e1fb6beac61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:05:56.483132 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1334/2659] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 15:05:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 15:05:56.491266 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a5c2c5-dd2e-4a0e-809e-b88e3beaf12f None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:05:56.491549 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a5c2c5-dd2e-4a0e-809e-b88e3beaf12f None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:05:56.491820 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a5c2c5-dd2e-4a0e-809e-b88e3beaf12f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:05:56.515076 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-a9a5c2c5-dd2e-4a0e-809e-b88e3beaf12f None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 15:05:56.516371 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-a9a5c2c5-dd2e-4a0e-809e-b88e3beaf12f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:05:56.525992 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1326/2660] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 15:05:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 15:05:56.530382 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acec56ad-a747-4601-a005-3e1fb6beac61 None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:05:56.543745 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acec56ad-a747-4601-a005-3e1fb6beac61 None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:05:56.548839 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432c37cb-edee-4832-ab9a-b4f155d14f51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:05:56.549447 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432c37cb-edee-4832-ab9a-b4f155d14f51 admin admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:05:56.549749 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432c37cb-edee-4832-ab9a-b4f155d14f51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:05:56.550015 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432c37cb-edee-4832-ab9a-b4f155d14f51 admin admin] PATH_INFO: `/v3/domains` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:05:56.550830 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432c37cb-edee-4832-ab9a-b4f155d14f51 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 15:05:56.551132 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432c37cb-edee-4832-ab9a-b4f155d14f51 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:05:56.552968 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432c37cb-edee-4832-ab9a-b4f155d14f51 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:05:56.567348 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1335/2661] 10.222.0.33 () {58 vars in 1153 bytes} [Thu Sep 1 15:05:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 15:05:56.571427 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a5c2c5-dd2e-4a0e-809e-b88e3beaf12f None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:05:56.584110 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a5c2c5-dd2e-4a0e-809e-b88e3beaf12f None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:05:56.589448 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b23d57-a717-440b-8ece-8e83ca8e0870 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:05:56.590166 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b23d57-a717-440b-8ece-8e83ca8e0870 admin admin] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:05:56.590563 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b23d57-a717-440b-8ece-8e83ca8e0870 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:05:56.590916 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b23d57-a717-440b-8ece-8e83ca8e0870 admin admin] PATH_INFO: `/v3/projects` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:05:56.591645 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b23d57-a717-440b-8ece-8e83ca8e0870 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:05:56.593650 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b23d57-a717-440b-8ece-8e83ca8e0870 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:05:56.612477 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 15:05:56.633883 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1327/2662] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Sep 1 15:05:56 2022] POST /identity/v3/projects => generated 367 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 15:05:56.634792 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432c37cb-edee-4832-ab9a-b4f155d14f51 admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:05:56.646530 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432c37cb-edee-4832-ab9a-b4f155d14f51 admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:05:56.713031 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a9dc02-773d-421b-af18-f21e632e6bb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:05:56.713949 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a9dc02-773d-421b-af18-f21e632e6bb0 admin admin] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:05:56.714231 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a9dc02-773d-421b-af18-f21e632e6bb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:05:56.714554 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a9dc02-773d-421b-af18-f21e632e6bb0 admin admin] PATH_INFO: `/v3/users` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:05:56.715187 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a9dc02-773d-421b-af18-f21e632e6bb0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:05:56.717243 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a9dc02-773d-421b-af18-f21e632e6bb0 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:05:56.745692 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1336/2663] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Sep 1 15:05:56 2022] POST /identity/v3/users => generated 284 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 15:05:56.749921 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b23d57-a717-440b-8ece-8e83ca8e0870 admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:05:56.766423 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b23d57-a717-440b-8ece-8e83ca8e0870 admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:05:56.770486 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3100cc-c645-466a-9db1-9618e09bb799 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:05:56.771308 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3100cc-c645-466a-9db1-9618e09bb799 admin admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:05:56.771759 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3100cc-c645-466a-9db1-9618e09bb799 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:05:56.772026 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3100cc-c645-466a-9db1-9618e09bb799 admin admin] PATH_INFO: `/v3/roles` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:05:56.772667 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3100cc-c645-466a-9db1-9618e09bb799 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:05:56.774336 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3100cc-c645-466a-9db1-9618e09bb799 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:05:56.784742 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1328/2664] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 1 15:05:56 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 15:05:56.789083 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a9dc02-773d-421b-af18-f21e632e6bb0 admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:05:56.800320 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a9dc02-773d-421b-af18-f21e632e6bb0 admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:05:56.806565 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1b467e-fac5-4929-8742-bb92aa25d41f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:05:56.807861 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1b467e-fac5-4929-8742-bb92aa25d41f admin admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:05:56.808185 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1b467e-fac5-4929-8742-bb92aa25d41f admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:05:56.808472 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1b467e-fac5-4929-8742-bb92aa25d41f admin admin] PATH_INFO: `/v3/roles` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:05:56.809117 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1b467e-fac5-4929-8742-bb92aa25d41f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:05:56.810745 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1b467e-fac5-4929-8742-bb92aa25d41f admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:05:56.821826 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1337/2665] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 1 15:05:56 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 15:05:56.826224 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3100cc-c645-466a-9db1-9618e09bb799 admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:05:56.836566 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3100cc-c645-466a-9db1-9618e09bb799 admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:05:56.841167 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52e5fdf-fd74-4385-9bc4-7fb1ace1485d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:05:56.841809 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52e5fdf-fd74-4385-9bc4-7fb1ace1485d admin admin] REQUEST_METHOD: `PUT` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:05:56.842088 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52e5fdf-fd74-4385-9bc4-7fb1ace1485d admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:05:56.842466 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52e5fdf-fd74-4385-9bc4-7fb1ace1485d admin admin] PATH_INFO: `/v3/projects/574f288bc7ee4bfd888909fb5608d925/users/8f44244a015140429d823171a8c299c0/roles/5d1feef0e13d4874928bd86258c4e3b2` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:05:56.857646 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52e5fdf-fd74-4385-9bc4-7fb1ace1485d admin admin] RBAC: Authorizing `identity:create_grant(project_id=574f288bc7ee4bfd888909fb5608d925, user_id=8f44244a015140429d823171a8c299c0, role_id=5d1feef0e13d4874928bd86258c4e3b2)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:05:56.860481 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52e5fdf-fd74-4385-9bc4-7fb1ace1485d admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:05:56.874335 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1329/2666] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 1 15:05:56 2022] PUT /identity/v3/projects/574f288bc7ee4bfd888909fb5608d925/users/8f44244a015140429d823171a8c299c0/roles/5d1feef0e13d4874928bd86258c4e3b2 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 15:05:56.885283 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191ca2d6-f7a8-4935-be36-a82aa0f096c0 None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:05:56.885597 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191ca2d6-f7a8-4935-be36-a82aa0f096c0 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:05:56.885900 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191ca2d6-f7a8-4935-be36-a82aa0f096c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:05:56.914597 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-191ca2d6-f7a8-4935-be36-a82aa0f096c0 None None] MFA Rules not processed for user `8f44244a015140429d823171a8c299c0`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 15:05:56.915817 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-191ca2d6-f7a8-4935-be36-a82aa0f096c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:05:56.957886 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 15:05:56.983179 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1338/2667] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 15:05:56 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 15:05:56.987659 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52e5fdf-fd74-4385-9bc4-7fb1ace1485d admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:05:57.049134 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52e5fdf-fd74-4385-9bc4-7fb1ace1485d admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:05:57.053742 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4b2c1b-7be3-4783-bb20-b8a26efec2aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:05:57.054418 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4b2c1b-7be3-4783-bb20-b8a26efec2aa admin admin] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:05:57.054701 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4b2c1b-7be3-4783-bb20-b8a26efec2aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:05:57.054972 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4b2c1b-7be3-4783-bb20-b8a26efec2aa admin admin] PATH_INFO: `/v3/projects` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:05:57.055626 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4b2c1b-7be3-4783-bb20-b8a26efec2aa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:05:57.057539 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4b2c1b-7be3-4783-bb20-b8a26efec2aa admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:05:57.082653 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1330/2668] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Sep 1 15:05:56 2022] POST /identity/v3/projects => generated 367 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 15:05:57.086853 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191ca2d6-f7a8-4935-be36-a82aa0f096c0 None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:05:57.098022 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191ca2d6-f7a8-4935-be36-a82aa0f096c0 None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:05:57.153204 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee27f0c-140e-4b65-ba61-7286fff98126 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:05:57.153808 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee27f0c-140e-4b65-ba61-7286fff98126 admin admin] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:05:57.154090 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee27f0c-140e-4b65-ba61-7286fff98126 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:05:57.154396 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee27f0c-140e-4b65-ba61-7286fff98126 admin admin] PATH_INFO: `/v3/users` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:05:57.155031 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee27f0c-140e-4b65-ba61-7286fff98126 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:05:57.156937 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee27f0c-140e-4b65-ba61-7286fff98126 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:05:57.177493 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1339/2669] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Sep 1 15:05:57 2022] POST /identity/v3/users => generated 290 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 15:05:57.181589 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4b2c1b-7be3-4783-bb20-b8a26efec2aa admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:05:57.191818 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4b2c1b-7be3-4783-bb20-b8a26efec2aa admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:05:57.196347 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae788720-fa38-4275-a115-f585390df9bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:05:57.196929 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae788720-fa38-4275-a115-f585390df9bb admin admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:05:57.197202 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae788720-fa38-4275-a115-f585390df9bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:05:57.197469 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae788720-fa38-4275-a115-f585390df9bb admin admin] PATH_INFO: `/v3/roles` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:05:57.198099 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae788720-fa38-4275-a115-f585390df9bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:05:57.199731 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae788720-fa38-4275-a115-f585390df9bb admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:05:57.210107 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1331/2670] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 1 15:05:57 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 15:05:57.214680 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee27f0c-140e-4b65-ba61-7286fff98126 admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:05:57.224511 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee27f0c-140e-4b65-ba61-7286fff98126 admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:05:57.229006 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa317b6-03b9-4edc-b87a-57a25175c587 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:05:57.230169 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa317b6-03b9-4edc-b87a-57a25175c587 admin admin] REQUEST_METHOD: `PUT` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:05:57.230475 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa317b6-03b9-4edc-b87a-57a25175c587 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:05:57.230745 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa317b6-03b9-4edc-b87a-57a25175c587 admin admin] PATH_INFO: `/v3/projects/e29c6d4aae3e4282ad20f30acdc318d8/users/858225b7497049238cb9434b481d0fd6/roles/2a2adb769e55476b82e5014897e0a9c0` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:05:57.245938 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa317b6-03b9-4edc-b87a-57a25175c587 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e29c6d4aae3e4282ad20f30acdc318d8, user_id=858225b7497049238cb9434b481d0fd6, role_id=2a2adb769e55476b82e5014897e0a9c0)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:05:57.248623 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa317b6-03b9-4edc-b87a-57a25175c587 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:05:57.261170 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1340/2671] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 1 15:05:57 2022] PUT /identity/v3/projects/e29c6d4aae3e4282ad20f30acdc318d8/users/858225b7497049238cb9434b481d0fd6/roles/2a2adb769e55476b82e5014897e0a9c0 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 15:05:57.265393 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae788720-fa38-4275-a115-f585390df9bb admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:05:57.275247 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae788720-fa38-4275-a115-f585390df9bb admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:05:57.330834 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8718ba-43c8-4b7f-8aca-1e102a63b208 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:05:57.331432 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8718ba-43c8-4b7f-8aca-1e102a63b208 admin admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:05:57.331711 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8718ba-43c8-4b7f-8aca-1e102a63b208 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:05:57.331993 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8718ba-43c8-4b7f-8aca-1e102a63b208 admin admin] PATH_INFO: `/v3/roles` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:05:57.332615 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8718ba-43c8-4b7f-8aca-1e102a63b208 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:05:57.334201 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8718ba-43c8-4b7f-8aca-1e102a63b208 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:05:57.344789 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1332/2672] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 1 15:05:57 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 15:05:57.348948 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa317b6-03b9-4edc-b87a-57a25175c587 admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:05:57.359027 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa317b6-03b9-4edc-b87a-57a25175c587 admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:05:57.363486 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bed62f8-45e8-4cc4-ad26-aba9606424b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:05:57.364115 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bed62f8-45e8-4cc4-ad26-aba9606424b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:05:57.364386 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bed62f8-45e8-4cc4-ad26-aba9606424b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:05:57.364649 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bed62f8-45e8-4cc4-ad26-aba9606424b8 admin admin] PATH_INFO: `/v3/projects/e29c6d4aae3e4282ad20f30acdc318d8/users/858225b7497049238cb9434b481d0fd6/roles/2a2adb769e55476b82e5014897e0a9c0` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:05:57.366589 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bed62f8-45e8-4cc4-ad26-aba9606424b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e29c6d4aae3e4282ad20f30acdc318d8, user_id=858225b7497049238cb9434b481d0fd6, role_id=2a2adb769e55476b82e5014897e0a9c0)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:05:57.369203 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bed62f8-45e8-4cc4-ad26-aba9606424b8 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:05:57.378943 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1341/2673] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 1 15:05:57 2022] PUT /identity/v3/projects/e29c6d4aae3e4282ad20f30acdc318d8/users/858225b7497049238cb9434b481d0fd6/roles/2a2adb769e55476b82e5014897e0a9c0 => generated 0 bytes in 30 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 15:05:57.390212 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6270ff6-0eac-4d0d-87c9-133302156470 None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:05:57.390535 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6270ff6-0eac-4d0d-87c9-133302156470 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:05:57.390815 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6270ff6-0eac-4d0d-87c9-133302156470 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:05:57.418533 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-e6270ff6-0eac-4d0d-87c9-133302156470 None None] MFA Rules not processed for user `858225b7497049238cb9434b481d0fd6`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 15:05:57.419744 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-e6270ff6-0eac-4d0d-87c9-133302156470 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:05:57.437875 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 15:05:57.487946 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1333/2674] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 15:05:57 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 15:05:57.499096 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bed62f8-45e8-4cc4-ad26-aba9606424b8 admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:05:57.500788 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-6bed62f8-45e8-4cc4-ad26-aba9606424b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:05:57.559665 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bed62f8-45e8-4cc4-ad26-aba9606424b8 admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:05:57.563250 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9340ee-cd8a-4cfe-8349-681dee5929b3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3b81739e34e4e648f88b6d73fcb5e46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:05:57.563841 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9340ee-cd8a-4cfe-8349-681dee5929b3 service cinder] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:05:57.564115 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9340ee-cd8a-4cfe-8349-681dee5929b3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:05:57.564382 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9340ee-cd8a-4cfe-8349-681dee5929b3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:05:57.573571 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9340ee-cd8a-4cfe-8349-681dee5929b3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:05:57.574994 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9340ee-cd8a-4cfe-8349-681dee5929b3 service cinder] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:05:57.584166 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1342/2675] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 15:05:57 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 15:05:57.665544 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6270ff6-0eac-4d0d-87c9-133302156470 None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:05:57.676101 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6270ff6-0eac-4d0d-87c9-133302156470 None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:05:57.679714 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48400db7-d272-4054-9832-4830aa9cda13 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3b81739e34e4e648f88b6d73fcb5e46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:05:57.680308 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48400db7-d272-4054-9832-4830aa9cda13 service cinder] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:05:57.680583 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48400db7-d272-4054-9832-4830aa9cda13 service cinder] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:05:57.680849 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48400db7-d272-4054-9832-4830aa9cda13 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:05:57.742767 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48400db7-d272-4054-9832-4830aa9cda13 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:05:57.744206 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48400db7-d272-4054-9832-4830aa9cda13 service cinder] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:05:57.753577 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1334/2676] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 15:05:57 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 15:06:01.879291 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9340ee-cd8a-4cfe-8349-681dee5929b3 service cinder] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:06:01.880906 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-7c9340ee-cd8a-4cfe-8349-681dee5929b3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:06:01.924245 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 15:06:01.948310 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9340ee-cd8a-4cfe-8349-681dee5929b3 service cinder] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:06:01.952788 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94d33d2-fb6b-41f1-903a-6c4d097a90df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:06:01.953402 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94d33d2-fb6b-41f1-903a-6c4d097a90df admin admin] REQUEST_METHOD: `DELETE` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:06:01.953675 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94d33d2-fb6b-41f1-903a-6c4d097a90df admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:06:01.953944 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94d33d2-fb6b-41f1-903a-6c4d097a90df admin admin] PATH_INFO: `/v3/users/b6682ccfe402445b82919b1530ac8c10` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:06:01.955073 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94d33d2-fb6b-41f1-903a-6c4d097a90df admin admin] RBAC: Authorizing `identity:delete_user(user_id=b6682ccfe402445b82919b1530ac8c10)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:06:01.956981 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94d33d2-fb6b-41f1-903a-6c4d097a90df admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:06:02.001028 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-e94d33d2-fb6b-41f1-903a-6c4d097a90df admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b6682ccfe402445b82919b1530ac8c10'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:06:02.008274 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-e94d33d2-fb6b-41f1-903a-6c4d097a90df admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b6682ccfe402445b82919b1530ac8c10'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:06:02.008886 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-e94d33d2-fb6b-41f1-903a-6c4d097a90df admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b6682ccfe402445b82919b1530ac8c10'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:06:02.020280 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-e94d33d2-fb6b-41f1-903a-6c4d097a90df admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b6682ccfe402445b82919b1530ac8c10'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:06:02.020896 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-e94d33d2-fb6b-41f1-903a-6c4d097a90df admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b6682ccfe402445b82919b1530ac8c10'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:06:02.032769 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1343/2677] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Sep 1 15:06:01 2022] DELETE /identity/v3/users/b6682ccfe402445b82919b1530ac8c10 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 15:06:02.204628 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48400db7-d272-4054-9832-4830aa9cda13 service cinder] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:06:02.216136 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48400db7-d272-4054-9832-4830aa9cda13 service cinder] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:06:02.271855 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d639d7-408d-4be7-9d61-6453da8d16dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:06:02.272479 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d639d7-408d-4be7-9d61-6453da8d16dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:06:02.272760 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d639d7-408d-4be7-9d61-6453da8d16dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:06:02.273029 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d639d7-408d-4be7-9d61-6453da8d16dc admin admin] PATH_INFO: `/v3/projects/9eb2f6885d9944bf90f2aa789a990749` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:06:02.274144 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d639d7-408d-4be7-9d61-6453da8d16dc admin admin] RBAC: Authorizing `identity:delete_project(project_id=9eb2f6885d9944bf90f2aa789a990749)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:06:02.276572 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d639d7-408d-4be7-9d61-6453da8d16dc admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:06:02.321662 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-e6d639d7-408d-4be7-9d61-6453da8d16dc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9eb2f6885d9944bf90f2aa789a990749', 'request_id': 'req-e6d639d7-408d-4be7-9d61-6453da8d16dc'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:06:02.328853 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-e6d639d7-408d-4be7-9d61-6453da8d16dc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9eb2f6885d9944bf90f2aa789a990749', 'request_id': 'req-e6d639d7-408d-4be7-9d61-6453da8d16dc'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:06:02.331529 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 15:06:02.337748 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-e6d639d7-408d-4be7-9d61-6453da8d16dc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9eb2f6885d9944bf90f2aa789a990749', 'request_id': 'req-e6d639d7-408d-4be7-9d61-6453da8d16dc'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:06:02.349347 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-e6d639d7-408d-4be7-9d61-6453da8d16dc admin admin] The token cache is being invalidate because project 9eb2f6885d9944bf90f2aa789a990749 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81341) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 15:06:02.349635 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-e6d639d7-408d-4be7-9d61-6453da8d16dc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:06:02.350281 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-e6d639d7-408d-4be7-9d61-6453da8d16dc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:06:02.351763 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1335/2678] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Sep 1 15:06:02 2022] DELETE /identity/v3/projects/9eb2f6885d9944bf90f2aa789a990749 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 15:06:02.356470 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94d33d2-fb6b-41f1-903a-6c4d097a90df admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:06:02.358106 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-e94d33d2-fb6b-41f1-903a-6c4d097a90df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:06:02.373337 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94d33d2-fb6b-41f1-903a-6c4d097a90df admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:06:02.438137 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f5bc27-eefe-40a4-a69f-ef8815477869 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:06:02.438830 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f5bc27-eefe-40a4-a69f-ef8815477869 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:06:02.439114 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f5bc27-eefe-40a4-a69f-ef8815477869 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:06:02.439391 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f5bc27-eefe-40a4-a69f-ef8815477869 admin admin] PATH_INFO: `/v3/users/497118c4126444398c23cd951eed7086` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:06:02.440709 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f5bc27-eefe-40a4-a69f-ef8815477869 admin admin] RBAC: Authorizing `identity:delete_user(user_id=497118c4126444398c23cd951eed7086)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:06:02.442732 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f5bc27-eefe-40a4-a69f-ef8815477869 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:06:02.481402 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d639d7-408d-4be7-9d61-6453da8d16dc admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:06:02.482844 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-e6d639d7-408d-4be7-9d61-6453da8d16dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:06:02.493055 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-49f5bc27-eefe-40a4-a69f-ef8815477869 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '497118c4126444398c23cd951eed7086'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:06:02.496801 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d639d7-408d-4be7-9d61-6453da8d16dc admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:06:02.499895 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-49f5bc27-eefe-40a4-a69f-ef8815477869 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '497118c4126444398c23cd951eed7086'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:06:02.500637 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-49f5bc27-eefe-40a4-a69f-ef8815477869 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '497118c4126444398c23cd951eed7086'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:06:02.502160 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7e225d-472e-4497-85b9-0507400a4195 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:06:02.502884 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7e225d-472e-4497-85b9-0507400a4195 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:06:02.503165 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7e225d-472e-4497-85b9-0507400a4195 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:06:02.503460 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7e225d-472e-4497-85b9-0507400a4195 admin admin] PATH_INFO: `/v3/users/8f44244a015140429d823171a8c299c0` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:06:02.504591 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7e225d-472e-4497-85b9-0507400a4195 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8f44244a015140429d823171a8c299c0)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:06:02.506378 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 15:06:02.506594 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7e225d-472e-4497-85b9-0507400a4195 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:06:02.515865 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-49f5bc27-eefe-40a4-a69f-ef8815477869 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '497118c4126444398c23cd951eed7086'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:06:02.516866 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-49f5bc27-eefe-40a4-a69f-ef8815477869 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '497118c4126444398c23cd951eed7086'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:06:02.530202 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1344/2679] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Sep 1 15:06:02 2022] DELETE /identity/v3/users/497118c4126444398c23cd951eed7086 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 15:06:02.556308 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-6a7e225d-472e-4497-85b9-0507400a4195 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8f44244a015140429d823171a8c299c0'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:06:02.563795 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-6a7e225d-472e-4497-85b9-0507400a4195 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8f44244a015140429d823171a8c299c0'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:06:02.564595 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-6a7e225d-472e-4497-85b9-0507400a4195 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8f44244a015140429d823171a8c299c0'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:06:02.580576 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-6a7e225d-472e-4497-85b9-0507400a4195 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8f44244a015140429d823171a8c299c0'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:06:02.581261 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-6a7e225d-472e-4497-85b9-0507400a4195 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8f44244a015140429d823171a8c299c0'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:06:02.593860 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1336/2680] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Sep 1 15:06:02 2022] DELETE /identity/v3/users/8f44244a015140429d823171a8c299c0 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 15:06:02.598379 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea46bcf-18e7-42d4-bde4-8bd599cd5d71 None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:06:02.598661 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea46bcf-18e7-42d4-bde4-8bd599cd5d71 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:06:02.598927 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea46bcf-18e7-42d4-bde4-8bd599cd5d71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:06:02.617491 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-9ea46bcf-18e7-42d4-bde4-8bd599cd5d71 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 15:06:02.618721 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-9ea46bcf-18e7-42d4-bde4-8bd599cd5d71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:06:02.681636 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1345/2681] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 15:06:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 15:06:02.692579 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7e225d-472e-4497-85b9-0507400a4195 admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:06:02.694017 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-6a7e225d-472e-4497-85b9-0507400a4195 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:06:02.710802 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea46bcf-18e7-42d4-bde4-8bd599cd5d71 None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:06:02.712230 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-9ea46bcf-18e7-42d4-bde4-8bd599cd5d71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:06:02.725088 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea46bcf-18e7-42d4-bde4-8bd599cd5d71 None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:06:02.729901 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fd5b9b-5ecf-4fa8-a2a7-6cf4a8f59081 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:06:02.730634 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fd5b9b-5ecf-4fa8-a2a7-6cf4a8f59081 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:06:02.730974 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fd5b9b-5ecf-4fa8-a2a7-6cf4a8f59081 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:06:02.731310 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fd5b9b-5ecf-4fa8-a2a7-6cf4a8f59081 admin admin] PATH_INFO: `/v3/projects/6aed9bea45b243d093faa957a8087fed` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:06:02.732646 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fd5b9b-5ecf-4fa8-a2a7-6cf4a8f59081 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6aed9bea45b243d093faa957a8087fed)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:06:02.734656 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fd5b9b-5ecf-4fa8-a2a7-6cf4a8f59081 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:06:02.773481 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7e225d-472e-4497-85b9-0507400a4195 admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:06:02.786674 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-45fd5b9b-5ecf-4fa8-a2a7-6cf4a8f59081 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6aed9bea45b243d093faa957a8087fed', 'request_id': 'req-45fd5b9b-5ecf-4fa8-a2a7-6cf4a8f59081'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:06:02.793098 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-45fd5b9b-5ecf-4fa8-a2a7-6cf4a8f59081 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6aed9bea45b243d093faa957a8087fed', 'request_id': 'req-45fd5b9b-5ecf-4fa8-a2a7-6cf4a8f59081'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:06:02.801635 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-45fd5b9b-5ecf-4fa8-a2a7-6cf4a8f59081 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6aed9bea45b243d093faa957a8087fed', 'request_id': 'req-45fd5b9b-5ecf-4fa8-a2a7-6cf4a8f59081'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:06:02.813834 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-45fd5b9b-5ecf-4fa8-a2a7-6cf4a8f59081 admin admin] The token cache is being invalidate because project 6aed9bea45b243d093faa957a8087fed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81342) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 15:06:02.814555 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-45fd5b9b-5ecf-4fa8-a2a7-6cf4a8f59081 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:06:02.815223 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-45fd5b9b-5ecf-4fa8-a2a7-6cf4a8f59081 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:06:02.817043 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1346/2682] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Sep 1 15:06:02 2022] DELETE /identity/v3/projects/6aed9bea45b243d093faa957a8087fed => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 15:06:02.821588 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4e06bc-eaef-4735-aca6-6900bdeaee03 None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:06:02.821871 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4e06bc-eaef-4735-aca6-6900bdeaee03 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:06:02.822150 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4e06bc-eaef-4735-aca6-6900bdeaee03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:06:02.822812 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ad74bf-9831-4f7e-bd43-e40ea53d4cd9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2790149639aa418bbc4e4982e93eda44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:06:02.823923 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ad74bf-9831-4f7e-bd43-e40ea53d4cd9 service neutron] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:06:02.824199 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ad74bf-9831-4f7e-bd43-e40ea53d4cd9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:06:02.824467 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ad74bf-9831-4f7e-bd43-e40ea53d4cd9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:06:02.826204 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-66ad74bf-9831-4f7e-bd43-e40ea53d4cd9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:06:02.842724 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-8f4e06bc-eaef-4735-aca6-6900bdeaee03 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 15:06:02.843975 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-8f4e06bc-eaef-4735-aca6-6900bdeaee03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:06:02.865036 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 15:06:02.899551 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 15:06:02.916678 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1347/2683] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 15:06:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 15:06:02.918157 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85487112-274b-4670-b2fa-2c4c901da16c None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:06:02.919529 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85487112-274b-4670-b2fa-2c4c901da16c None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:06:02.919742 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85487112-274b-4670-b2fa-2c4c901da16c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:06:02.919870 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ad74bf-9831-4f7e-bd43-e40ea53d4cd9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:06:02.921756 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ad74bf-9831-4f7e-bd43-e40ea53d4cd9 service neutron] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:06:02.933487 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1337/2684] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 15:06:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 241 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 15:06:02.943340 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-85487112-274b-4670-b2fa-2c4c901da16c None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 15:06:02.944719 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-85487112-274b-4670-b2fa-2c4c901da16c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:06:02.954022 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1348/2685] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 15:06:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 15:06:02.958439 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ad74bf-9831-4f7e-bd43-e40ea53d4cd9 service neutron] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:06:02.969224 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ad74bf-9831-4f7e-bd43-e40ea53d4cd9 service neutron] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:06:02.974985 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbea513-6d3c-4112-8f29-41f2bbc77c26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:06:02.975593 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbea513-6d3c-4112-8f29-41f2bbc77c26 admin admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:06:02.975872 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbea513-6d3c-4112-8f29-41f2bbc77c26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:06:02.976143 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbea513-6d3c-4112-8f29-41f2bbc77c26 admin admin] PATH_INFO: `/v3/domains` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:06:02.976821 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbea513-6d3c-4112-8f29-41f2bbc77c26 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81341) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 15:06:02.977243 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbea513-6d3c-4112-8f29-41f2bbc77c26 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:06:02.978919 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbea513-6d3c-4112-8f29-41f2bbc77c26 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:06:02.992027 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1338/2686] 10.222.0.33 () {58 vars in 1153 bytes} [Thu Sep 1 15:06:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 15:06:02.996982 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85487112-274b-4670-b2fa-2c4c901da16c None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:06:03.006828 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85487112-274b-4670-b2fa-2c4c901da16c None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:06:03.012731 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f691f0-40ad-4dc5-a4e7-9f27de648abe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:06:03.013367 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f691f0-40ad-4dc5-a4e7-9f27de648abe admin admin] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:06:03.013637 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f691f0-40ad-4dc5-a4e7-9f27de648abe admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:06:03.013904 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f691f0-40ad-4dc5-a4e7-9f27de648abe admin admin] PATH_INFO: `/v3/projects` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:06:03.014600 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f691f0-40ad-4dc5-a4e7-9f27de648abe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:06:03.016539 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f691f0-40ad-4dc5-a4e7-9f27de648abe admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:06:03.048749 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1349/2687] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Sep 1 15:06:02 2022] POST /identity/v3/projects => generated 359 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 15:06:03.050144 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbea513-6d3c-4112-8f29-41f2bbc77c26 admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:06:03.061394 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbea513-6d3c-4112-8f29-41f2bbc77c26 admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:06:03.124788 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94447a0-8918-49ba-99d4-3e4668c2b2b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:06:03.125392 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94447a0-8918-49ba-99d4-3e4668c2b2b9 admin admin] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:06:03.125670 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94447a0-8918-49ba-99d4-3e4668c2b2b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:06:03.125942 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94447a0-8918-49ba-99d4-3e4668c2b2b9 admin admin] PATH_INFO: `/v3/users` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:06:03.126611 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94447a0-8918-49ba-99d4-3e4668c2b2b9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:06:03.128551 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94447a0-8918-49ba-99d4-3e4668c2b2b9 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:06:03.151037 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1339/2688] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Sep 1 15:06:03 2022] POST /identity/v3/users => generated 280 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 15:06:03.155022 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f691f0-40ad-4dc5-a4e7-9f27de648abe admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:06:03.166990 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f691f0-40ad-4dc5-a4e7-9f27de648abe admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:06:03.172086 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1aeac1-8c0e-4ee1-b860-63eef98a7c24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:06:03.172704 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1aeac1-8c0e-4ee1-b860-63eef98a7c24 admin admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:06:03.172960 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1aeac1-8c0e-4ee1-b860-63eef98a7c24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:06:03.173224 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1aeac1-8c0e-4ee1-b860-63eef98a7c24 admin admin] PATH_INFO: `/v3/roles` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:06:03.173862 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1aeac1-8c0e-4ee1-b860-63eef98a7c24 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:06:03.175488 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1aeac1-8c0e-4ee1-b860-63eef98a7c24 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:06:03.186595 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1350/2689] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 1 15:06:03 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 15:06:03.190656 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94447a0-8918-49ba-99d4-3e4668c2b2b9 admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:06:03.203859 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94447a0-8918-49ba-99d4-3e4668c2b2b9 admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:06:03.208995 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e157e6b-2b4b-4b5d-bfde-665e4238d786 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:06:03.209912 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e157e6b-2b4b-4b5d-bfde-665e4238d786 admin admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:06:03.210331 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e157e6b-2b4b-4b5d-bfde-665e4238d786 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:06:03.210609 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e157e6b-2b4b-4b5d-bfde-665e4238d786 admin admin] PATH_INFO: `/v3/roles` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:06:03.211260 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e157e6b-2b4b-4b5d-bfde-665e4238d786 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:06:03.213032 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e157e6b-2b4b-4b5d-bfde-665e4238d786 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:06:03.221503 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1aeac1-8c0e-4ee1-b860-63eef98a7c24 admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:06:03.223436 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-9e1aeac1-8c0e-4ee1-b860-63eef98a7c24 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:06:03.225061 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1340/2690] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 1 15:06:03 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 15:06:03.228943 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e157e6b-2b4b-4b5d-bfde-665e4238d786 admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:06:03.238894 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1aeac1-8c0e-4ee1-b860-63eef98a7c24 admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:06:03.239504 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e157e6b-2b4b-4b5d-bfde-665e4238d786 admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:06:03.244388 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4bb05e-fd1a-448f-89bc-e387962c6ed7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:06:03.245272 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f237d75d-47e2-4323-b64a-8757ac0021a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:06:03.245796 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4bb05e-fd1a-448f-89bc-e387962c6ed7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:06:03.245937 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f237d75d-47e2-4323-b64a-8757ac0021a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:06:03.246196 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4bb05e-fd1a-448f-89bc-e387962c6ed7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:06:03.246362 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f237d75d-47e2-4323-b64a-8757ac0021a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:06:03.246507 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f237d75d-47e2-4323-b64a-8757ac0021a3 admin admin] PATH_INFO: `/v3/projects/5e386075738c4ff5a049132c2d83dd35/users/4295a74e11584351b3b19e031b2673c7/roles/5d1feef0e13d4874928bd86258c4e3b2` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:06:03.246666 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4bb05e-fd1a-448f-89bc-e387962c6ed7 admin admin] PATH_INFO: `/v3/projects/574f288bc7ee4bfd888909fb5608d925` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:06:03.247814 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4bb05e-fd1a-448f-89bc-e387962c6ed7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=574f288bc7ee4bfd888909fb5608d925)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:06:03.249828 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4bb05e-fd1a-448f-89bc-e387962c6ed7 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:06:03.264476 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f237d75d-47e2-4323-b64a-8757ac0021a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e386075738c4ff5a049132c2d83dd35, user_id=4295a74e11584351b3b19e031b2673c7, role_id=5d1feef0e13d4874928bd86258c4e3b2)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:06:03.267275 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f237d75d-47e2-4323-b64a-8757ac0021a3 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:06:03.281452 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1341/2691] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 1 15:06:03 2022] PUT /identity/v3/projects/5e386075738c4ff5a049132c2d83dd35/users/4295a74e11584351b3b19e031b2673c7/roles/5d1feef0e13d4874928bd86258c4e3b2 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 15:06:03.291484 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d78fb4e-22b1-404a-9b17-79138a953756 None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:06:03.292362 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d78fb4e-22b1-404a-9b17-79138a953756 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:06:03.292636 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d78fb4e-22b1-404a-9b17-79138a953756 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:06:03.299743 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-6f4bb05e-fd1a-448f-89bc-e387962c6ed7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '574f288bc7ee4bfd888909fb5608d925', 'request_id': 'req-6f4bb05e-fd1a-448f-89bc-e387962c6ed7'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:06:03.307214 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-6f4bb05e-fd1a-448f-89bc-e387962c6ed7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '574f288bc7ee4bfd888909fb5608d925', 'request_id': 'req-6f4bb05e-fd1a-448f-89bc-e387962c6ed7'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:06:03.315741 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-6f4bb05e-fd1a-448f-89bc-e387962c6ed7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '574f288bc7ee4bfd888909fb5608d925', 'request_id': 'req-6f4bb05e-fd1a-448f-89bc-e387962c6ed7'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:06:03.321644 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-0d78fb4e-22b1-404a-9b17-79138a953756 None None] MFA Rules not processed for user `4295a74e11584351b3b19e031b2673c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 15:06:03.322943 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-0d78fb4e-22b1-404a-9b17-79138a953756 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:06:03.326730 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-6f4bb05e-fd1a-448f-89bc-e387962c6ed7 admin admin] The token cache is being invalidate because project 574f288bc7ee4bfd888909fb5608d925 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81342) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 15:06:03.327022 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-6f4bb05e-fd1a-448f-89bc-e387962c6ed7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:06:03.327681 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-6f4bb05e-fd1a-448f-89bc-e387962c6ed7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:06:03.329177 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1351/2692] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Sep 1 15:06:03 2022] DELETE /identity/v3/projects/574f288bc7ee4bfd888909fb5608d925 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 15:06:03.333122 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4bb05e-fd1a-448f-89bc-e387962c6ed7 admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:06:03.334454 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-6f4bb05e-fd1a-448f-89bc-e387962c6ed7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:06:03.380617 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 15:06:03.393309 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1342/2693] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 15:06:03 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 15:06:03.420687 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4bb05e-fd1a-448f-89bc-e387962c6ed7 admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:06:03.425556 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88dd86f0-fc94-4431-9b8f-8870e7bfb828 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:06:03.426189 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dd86f0-fc94-4431-9b8f-8870e7bfb828 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:06:03.426498 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dd86f0-fc94-4431-9b8f-8870e7bfb828 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:06:03.426770 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dd86f0-fc94-4431-9b8f-8870e7bfb828 admin admin] PATH_INFO: `/v3/users/858225b7497049238cb9434b481d0fd6` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:06:03.427906 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88dd86f0-fc94-4431-9b8f-8870e7bfb828 admin admin] RBAC: Authorizing `identity:delete_user(user_id=858225b7497049238cb9434b481d0fd6)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:06:03.429820 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88dd86f0-fc94-4431-9b8f-8870e7bfb828 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:06:03.430306 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d78fb4e-22b1-404a-9b17-79138a953756 None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:06:03.431719 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-0d78fb4e-22b1-404a-9b17-79138a953756 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:06:03.475694 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=84,keystone:INSERT=2,keystone:DELETE=14 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 15:06:03.477248 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-88dd86f0-fc94-4431-9b8f-8870e7bfb828 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '858225b7497049238cb9434b481d0fd6'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:06:03.486756 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-88dd86f0-fc94-4431-9b8f-8870e7bfb828 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '858225b7497049238cb9434b481d0fd6'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:06:03.487861 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-88dd86f0-fc94-4431-9b8f-8870e7bfb828 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '858225b7497049238cb9434b481d0fd6'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:06:03.492999 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d78fb4e-22b1-404a-9b17-79138a953756 None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:06:03.497284 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8baa115d-28eb-448a-a460-ba737e06a206 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3b81739e34e4e648f88b6d73fcb5e46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:06:03.497888 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8baa115d-28eb-448a-a460-ba737e06a206 service cinder] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:06:03.498171 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8baa115d-28eb-448a-a460-ba737e06a206 service cinder] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:06:03.498474 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8baa115d-28eb-448a-a460-ba737e06a206 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:06:03.502105 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-88dd86f0-fc94-4431-9b8f-8870e7bfb828 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '858225b7497049238cb9434b481d0fd6'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:06:03.502806 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-88dd86f0-fc94-4431-9b8f-8870e7bfb828 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '858225b7497049238cb9434b481d0fd6'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:06:03.506819 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8baa115d-28eb-448a-a460-ba737e06a206 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:06:03.508294 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8baa115d-28eb-448a-a460-ba737e06a206 service cinder] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:06:03.515714 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1352/2694] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Sep 1 15:06:03 2022] DELETE /identity/v3/users/858225b7497049238cb9434b481d0fd6 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 15:06:03.517018 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1343/2695] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 15:06:03 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 15:06:03.532940 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88dd86f0-fc94-4431-9b8f-8870e7bfb828 admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:06:03.534541 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-88dd86f0-fc94-4431-9b8f-8870e7bfb828 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:06:03.610592 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88dd86f0-fc94-4431-9b8f-8870e7bfb828 admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:06:03.615387 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebb050c-9080-4e5b-95c2-a08304c8a5d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:06:03.616115 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebb050c-9080-4e5b-95c2-a08304c8a5d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:06:03.616461 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebb050c-9080-4e5b-95c2-a08304c8a5d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:06:03.616806 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebb050c-9080-4e5b-95c2-a08304c8a5d5 admin admin] PATH_INFO: `/v3/users/4295a74e11584351b3b19e031b2673c7` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:06:03.618021 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebb050c-9080-4e5b-95c2-a08304c8a5d5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4295a74e11584351b3b19e031b2673c7)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:06:03.620628 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebb050c-9080-4e5b-95c2-a08304c8a5d5 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:06:03.683521 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-5ebb050c-9080-4e5b-95c2-a08304c8a5d5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4295a74e11584351b3b19e031b2673c7'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:06:03.692333 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-5ebb050c-9080-4e5b-95c2-a08304c8a5d5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4295a74e11584351b3b19e031b2673c7'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:06:03.692992 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-5ebb050c-9080-4e5b-95c2-a08304c8a5d5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4295a74e11584351b3b19e031b2673c7'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:06:03.705414 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-5ebb050c-9080-4e5b-95c2-a08304c8a5d5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4295a74e11584351b3b19e031b2673c7'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:06:03.706228 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-5ebb050c-9080-4e5b-95c2-a08304c8a5d5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4295a74e11584351b3b19e031b2673c7'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:06:03.719255 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1353/2696] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Sep 1 15:06:03 2022] DELETE /identity/v3/users/4295a74e11584351b3b19e031b2673c7 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 15:06:03.723711 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815b0543-4710-4da8-8dbf-ee6f40721f6a None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:06:03.723999 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815b0543-4710-4da8-8dbf-ee6f40721f6a None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:06:03.724273 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815b0543-4710-4da8-8dbf-ee6f40721f6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:06:03.744213 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-815b0543-4710-4da8-8dbf-ee6f40721f6a None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 15:06:03.745469 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-815b0543-4710-4da8-8dbf-ee6f40721f6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:06:03.807219 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1344/2697] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 15:06:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 15:06:03.816579 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebb050c-9080-4e5b-95c2-a08304c8a5d5 admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:06:03.817745 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815b0543-4710-4da8-8dbf-ee6f40721f6a None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:06:03.818260 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-5ebb050c-9080-4e5b-95c2-a08304c8a5d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:06:03.819416 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-815b0543-4710-4da8-8dbf-ee6f40721f6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:06:03.847202 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebb050c-9080-4e5b-95c2-a08304c8a5d5 admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:06:03.852648 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a965eea7-404c-4938-adb5-3b1effd46cf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:06:03.853313 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a965eea7-404c-4938-adb5-3b1effd46cf0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:06:03.853591 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a965eea7-404c-4938-adb5-3b1effd46cf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:06:03.853866 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a965eea7-404c-4938-adb5-3b1effd46cf0 admin admin] PATH_INFO: `/v3/projects/e29c6d4aae3e4282ad20f30acdc318d8` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:06:03.855692 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a965eea7-404c-4938-adb5-3b1effd46cf0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e29c6d4aae3e4282ad20f30acdc318d8)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:06:03.857637 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a965eea7-404c-4938-adb5-3b1effd46cf0 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:06:03.878190 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815b0543-4710-4da8-8dbf-ee6f40721f6a None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:06:03.881982 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375df9af-4a78-47c3-890d-2b15a641339b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2790149639aa418bbc4e4982e93eda44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:06:03.882620 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375df9af-4a78-47c3-890d-2b15a641339b service neutron] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:06:03.882904 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375df9af-4a78-47c3-890d-2b15a641339b service neutron] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:06:03.883179 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375df9af-4a78-47c3-890d-2b15a641339b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:06:03.893270 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375df9af-4a78-47c3-890d-2b15a641339b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:06:03.894770 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375df9af-4a78-47c3-890d-2b15a641339b service neutron] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:06:03.896911 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=82,keystone:INSERT=2,keystone:DELETE=16 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 15:06:03.905560 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1345/2698] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 15:06:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 15:06:03.908189 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-a965eea7-404c-4938-adb5-3b1effd46cf0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e29c6d4aae3e4282ad20f30acdc318d8', 'request_id': 'req-a965eea7-404c-4938-adb5-3b1effd46cf0'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:06:03.914561 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-a965eea7-404c-4938-adb5-3b1effd46cf0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e29c6d4aae3e4282ad20f30acdc318d8', 'request_id': 'req-a965eea7-404c-4938-adb5-3b1effd46cf0'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:06:03.923515 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-a965eea7-404c-4938-adb5-3b1effd46cf0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e29c6d4aae3e4282ad20f30acdc318d8', 'request_id': 'req-a965eea7-404c-4938-adb5-3b1effd46cf0'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:06:03.934405 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-a965eea7-404c-4938-adb5-3b1effd46cf0 admin admin] The token cache is being invalidate because project e29c6d4aae3e4282ad20f30acdc318d8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81342) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 15:06:03.934713 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-a965eea7-404c-4938-adb5-3b1effd46cf0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:06:03.935343 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-a965eea7-404c-4938-adb5-3b1effd46cf0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:06:03.936863 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1354/2699] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Sep 1 15:06:03 2022] DELETE /identity/v3/projects/e29c6d4aae3e4282ad20f30acdc318d8 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 15:06:03.941767 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f788cf-02c5-4318-9c9a-d08233e57db3 None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:06:03.942193 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f788cf-02c5-4318-9c9a-d08233e57db3 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:06:03.942382 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f788cf-02c5-4318-9c9a-d08233e57db3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:06:03.964309 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-53f788cf-02c5-4318-9c9a-d08233e57db3 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 15:06:03.965536 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-53f788cf-02c5-4318-9c9a-d08233e57db3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:06:04.002453 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 15:06:04.030139 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1346/2700] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 15:06:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 15:06:04.037955 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d9b42c-7a5b-45dc-8f0b-31406feb0259 None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:06:04.038258 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d9b42c-7a5b-45dc-8f0b-31406feb0259 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:06:04.038544 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d9b42c-7a5b-45dc-8f0b-31406feb0259 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:06:04.059629 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-85d9b42c-7a5b-45dc-8f0b-31406feb0259 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 15:06:04.061054 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-85d9b42c-7a5b-45dc-8f0b-31406feb0259 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:06:04.070666 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1355/2701] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 15:06:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 15:06:04.075676 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f788cf-02c5-4318-9c9a-d08233e57db3 None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:06:04.086416 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f788cf-02c5-4318-9c9a-d08233e57db3 None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:06:04.091894 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0cce02-2c1d-4bdb-b993-d3af74fffb1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:06:04.092491 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0cce02-2c1d-4bdb-b993-d3af74fffb1f admin admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:06:04.092772 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0cce02-2c1d-4bdb-b993-d3af74fffb1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:06:04.093358 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0cce02-2c1d-4bdb-b993-d3af74fffb1f admin admin] PATH_INFO: `/v3/domains` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:06:04.093687 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0cce02-2c1d-4bdb-b993-d3af74fffb1f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81341) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 15:06:04.093988 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0cce02-2c1d-4bdb-b993-d3af74fffb1f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:06:04.095656 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0cce02-2c1d-4bdb-b993-d3af74fffb1f admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:06:04.107713 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1347/2702] 10.222.0.33 () {58 vars in 1153 bytes} [Thu Sep 1 15:06:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 15:06:04.112032 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d9b42c-7a5b-45dc-8f0b-31406feb0259 None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:06:04.122001 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d9b42c-7a5b-45dc-8f0b-31406feb0259 None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:06:04.126651 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bc38f7-ce24-4d3d-9790-64e3ec44e8c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:06:04.127269 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bc38f7-ce24-4d3d-9790-64e3ec44e8c2 admin admin] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:06:04.127542 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bc38f7-ce24-4d3d-9790-64e3ec44e8c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:06:04.127811 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bc38f7-ce24-4d3d-9790-64e3ec44e8c2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:06:04.128434 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bc38f7-ce24-4d3d-9790-64e3ec44e8c2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:06:04.130469 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bc38f7-ce24-4d3d-9790-64e3ec44e8c2 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:06:04.161524 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1356/2703] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Sep 1 15:06:04 2022] POST /identity/v3/projects => generated 385 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 15:06:04.165745 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0cce02-2c1d-4bdb-b993-d3af74fffb1f admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:06:04.177439 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0cce02-2c1d-4bdb-b993-d3af74fffb1f admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:06:04.191143 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bc38f7-ce24-4d3d-9790-64e3ec44e8c2 admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:06:04.193176 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-66bc38f7-ce24-4d3d-9790-64e3ec44e8c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:06:04.250811 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc14113d-6ca5-435e-a86a-5d7921ba59ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:06:04.251402 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc14113d-6ca5-435e-a86a-5d7921ba59ea admin admin] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:06:04.251681 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc14113d-6ca5-435e-a86a-5d7921ba59ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:06:04.251946 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc14113d-6ca5-435e-a86a-5d7921ba59ea admin admin] PATH_INFO: `/v3/users` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:06:04.252645 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc14113d-6ca5-435e-a86a-5d7921ba59ea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:06:04.254545 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc14113d-6ca5-435e-a86a-5d7921ba59ea admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:06:04.270763 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bc38f7-ce24-4d3d-9790-64e3ec44e8c2 admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:06:04.275772 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d615ef62-89e4-40ba-81ff-dc1911f3df6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:06:04.277092 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d615ef62-89e4-40ba-81ff-dc1911f3df6c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:06:04.277444 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d615ef62-89e4-40ba-81ff-dc1911f3df6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:06:04.277633 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1348/2704] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Sep 1 15:06:04 2022] POST /identity/v3/users => generated 293 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 15:06:04.278055 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d615ef62-89e4-40ba-81ff-dc1911f3df6c admin admin] PATH_INFO: `/v3/projects/5e386075738c4ff5a049132c2d83dd35` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:06:04.279335 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d615ef62-89e4-40ba-81ff-dc1911f3df6c admin admin] RBAC: Authorizing `identity:delete_project(project_id=5e386075738c4ff5a049132c2d83dd35)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:06:04.281240 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d615ef62-89e4-40ba-81ff-dc1911f3df6c admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:06:04.281653 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc14113d-6ca5-435e-a86a-5d7921ba59ea admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:06:04.291472 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc14113d-6ca5-435e-a86a-5d7921ba59ea admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:06:04.296554 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9d46cc-77e9-48bd-a132-638bbb2cbc50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:06:04.297144 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9d46cc-77e9-48bd-a132-638bbb2cbc50 admin admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:06:04.297425 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9d46cc-77e9-48bd-a132-638bbb2cbc50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:06:04.297782 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9d46cc-77e9-48bd-a132-638bbb2cbc50 admin admin] PATH_INFO: `/v3/roles` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:06:04.298360 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9d46cc-77e9-48bd-a132-638bbb2cbc50 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:06:04.299968 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9d46cc-77e9-48bd-a132-638bbb2cbc50 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:06:04.311457 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1349/2705] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 1 15:06:04 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 15:06:04.315426 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9d46cc-77e9-48bd-a132-638bbb2cbc50 admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:06:04.326063 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9d46cc-77e9-48bd-a132-638bbb2cbc50 admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:06:04.328992 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-d615ef62-89e4-40ba-81ff-dc1911f3df6c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5e386075738c4ff5a049132c2d83dd35', 'request_id': 'req-d615ef62-89e4-40ba-81ff-dc1911f3df6c'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:06:04.335580 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-d615ef62-89e4-40ba-81ff-dc1911f3df6c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5e386075738c4ff5a049132c2d83dd35', 'request_id': 'req-d615ef62-89e4-40ba-81ff-dc1911f3df6c'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:06:04.343058 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-d615ef62-89e4-40ba-81ff-dc1911f3df6c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5e386075738c4ff5a049132c2d83dd35', 'request_id': 'req-d615ef62-89e4-40ba-81ff-dc1911f3df6c'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:06:04.354890 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-d615ef62-89e4-40ba-81ff-dc1911f3df6c admin admin] The token cache is being invalidate because project 5e386075738c4ff5a049132c2d83dd35 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81342) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 15:06:04.355357 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-d615ef62-89e4-40ba-81ff-dc1911f3df6c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:06:04.356014 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-d615ef62-89e4-40ba-81ff-dc1911f3df6c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:06:04.357546 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1357/2706] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Sep 1 15:06:04 2022] DELETE /identity/v3/projects/5e386075738c4ff5a049132c2d83dd35 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 15:06:04.362157 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0ff797-6753-4135-af83-3ca145025128 None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:06:04.362476 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0ff797-6753-4135-af83-3ca145025128 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:06:04.362743 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0ff797-6753-4135-af83-3ca145025128 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:06:04.382051 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-1e0ff797-6753-4135-af83-3ca145025128 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 15:06:04.383311 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-1e0ff797-6753-4135-af83-3ca145025128 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:06:04.388719 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112808b8-1050-458d-ab4a-984071e1ecbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:06:04.389325 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112808b8-1050-458d-ab4a-984071e1ecbf admin admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:06:04.389607 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112808b8-1050-458d-ab4a-984071e1ecbf admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:06:04.389877 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112808b8-1050-458d-ab4a-984071e1ecbf admin admin] PATH_INFO: `/v3/roles` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:06:04.390540 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112808b8-1050-458d-ab4a-984071e1ecbf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:06:04.391866 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1358/2707] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 15:06:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 15:06:04.392135 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112808b8-1050-458d-ab4a-984071e1ecbf admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:06:04.399339 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8097bab2-2871-46f0-a8ff-683039cd4fbd None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:06:04.399623 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8097bab2-2871-46f0-a8ff-683039cd4fbd None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:06:04.399895 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8097bab2-2871-46f0-a8ff-683039cd4fbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:06:04.402928 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1350/2708] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 1 15:06:04 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 15:06:04.406712 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112808b8-1050-458d-ab4a-984071e1ecbf admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:06:04.408347 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-112808b8-1050-458d-ab4a-984071e1ecbf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:06:04.419639 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-8097bab2-2871-46f0-a8ff-683039cd4fbd None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 15:06:04.420573 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112808b8-1050-458d-ab4a-984071e1ecbf admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:06:04.421071 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-8097bab2-2871-46f0-a8ff-683039cd4fbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:06:04.425147 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8492715-142b-443b-ac09-984fbf08a275 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:06:04.425777 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8492715-142b-443b-ac09-984fbf08a275 admin admin] REQUEST_METHOD: `PUT` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:06:04.426057 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8492715-142b-443b-ac09-984fbf08a275 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:06:04.426359 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8492715-142b-443b-ac09-984fbf08a275 admin admin] PATH_INFO: `/v3/projects/37c8077b51324140ad47fbdda97f382b/users/b3bf16245bd74a77928ee9822a4b508d/roles/5d1feef0e13d4874928bd86258c4e3b2` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:06:04.428867 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1359/2709] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 15:06:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 15:06:04.433504 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8097bab2-2871-46f0-a8ff-683039cd4fbd None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:06:04.441404 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8492715-142b-443b-ac09-984fbf08a275 admin admin] RBAC: Authorizing `identity:create_grant(project_id=37c8077b51324140ad47fbdda97f382b, user_id=b3bf16245bd74a77928ee9822a4b508d, role_id=5d1feef0e13d4874928bd86258c4e3b2)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:06:04.442482 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8097bab2-2871-46f0-a8ff-683039cd4fbd None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:06:04.444093 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8492715-142b-443b-ac09-984fbf08a275 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:06:04.448632 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244a8091-4e7a-4041-bbc7-7119818d2161 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:06:04.449288 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244a8091-4e7a-4041-bbc7-7119818d2161 admin admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:06:04.449563 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244a8091-4e7a-4041-bbc7-7119818d2161 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:06:04.449835 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244a8091-4e7a-4041-bbc7-7119818d2161 admin admin] PATH_INFO: `/v3/domains` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:06:04.450546 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244a8091-4e7a-4041-bbc7-7119818d2161 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 15:06:04.450846 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244a8091-4e7a-4041-bbc7-7119818d2161 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:06:04.452431 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244a8091-4e7a-4041-bbc7-7119818d2161 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:06:04.457733 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1351/2710] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 1 15:06:04 2022] PUT /identity/v3/projects/37c8077b51324140ad47fbdda97f382b/users/b3bf16245bd74a77928ee9822a4b508d/roles/5d1feef0e13d4874928bd86258c4e3b2 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 15:06:04.467707 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 15:06:04.468934 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9da47d-a670-4bb3-b837-6e201a96eeb8 None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:06:04.469301 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9da47d-a670-4bb3-b837-6e201a96eeb8 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:06:04.470270 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9da47d-a670-4bb3-b837-6e201a96eeb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:06:04.474765 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1360/2711] 10.222.0.33 () {58 vars in 1153 bytes} [Thu Sep 1 15:06:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 15:06:04.476387 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244a8091-4e7a-4041-bbc7-7119818d2161 admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:06:04.486270 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244a8091-4e7a-4041-bbc7-7119818d2161 admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:06:04.499353 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-df9da47d-a670-4bb3-b837-6e201a96eeb8 None None] MFA Rules not processed for user `b3bf16245bd74a77928ee9822a4b508d`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 15:06:04.500567 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-df9da47d-a670-4bb3-b837-6e201a96eeb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:06:04.505725 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 15:06:04.544169 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9089fd-53b4-415c-a47e-01c151d9ed90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:06:04.544774 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9089fd-53b4-415c-a47e-01c151d9ed90 admin admin] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:06:04.545050 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9089fd-53b4-415c-a47e-01c151d9ed90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:06:04.545578 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9089fd-53b4-415c-a47e-01c151d9ed90 admin admin] PATH_INFO: `/v3/projects` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:06:04.546204 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9089fd-53b4-415c-a47e-01c151d9ed90 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:06:04.548204 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9089fd-53b4-415c-a47e-01c151d9ed90 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:06:04.567999 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1352/2712] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 15:06:04 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 15:06:04.573471 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9da47d-a670-4bb3-b837-6e201a96eeb8 None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:06:04.579683 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1361/2713] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Sep 1 15:06:04 2022] POST /identity/v3/projects => generated 399 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 15:06:04.580635 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9089fd-53b4-415c-a47e-01c151d9ed90 admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:06:04.584520 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9da47d-a670-4bb3-b837-6e201a96eeb8 None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:06:04.640532 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9089fd-53b4-415c-a47e-01c151d9ed90 admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:06:04.640861 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d9ff6e-565a-47b3-818f-2b9746ae01a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:06:04.641458 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d9ff6e-565a-47b3-818f-2b9746ae01a9 admin admin] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:06:04.641731 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d9ff6e-565a-47b3-818f-2b9746ae01a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:06:04.642010 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d9ff6e-565a-47b3-818f-2b9746ae01a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:06:04.642673 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d9ff6e-565a-47b3-818f-2b9746ae01a9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:06:04.644595 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d9ff6e-565a-47b3-818f-2b9746ae01a9 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:06:04.645312 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbf7d1b-7aef-4a45-8c69-7a14e939db44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:06:04.645906 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbf7d1b-7aef-4a45-8c69-7a14e939db44 admin admin] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:06:04.646193 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbf7d1b-7aef-4a45-8c69-7a14e939db44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:06:04.646548 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbf7d1b-7aef-4a45-8c69-7a14e939db44 admin admin] PATH_INFO: `/v3/users` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:06:04.647170 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbf7d1b-7aef-4a45-8c69-7a14e939db44 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:06:04.649070 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbf7d1b-7aef-4a45-8c69-7a14e939db44 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:06:04.677175 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1362/2714] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Sep 1 15:06:04 2022] POST /identity/v3/users => generated 300 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 15:06:04.678863 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1353/2715] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Sep 1 15:06:04 2022] POST /identity/v3/projects => generated 385 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 15:06:04.683453 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbf7d1b-7aef-4a45-8c69-7a14e939db44 admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:06:04.683584 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d9ff6e-565a-47b3-818f-2b9746ae01a9 admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:06:04.693339 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbf7d1b-7aef-4a45-8c69-7a14e939db44 admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:06:04.753065 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d9ff6e-565a-47b3-818f-2b9746ae01a9 admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:06:04.753541 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76f0025-ffb9-425e-ae69-2f1abb5e655e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:06:04.754149 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76f0025-ffb9-425e-ae69-2f1abb5e655e admin admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:06:04.754467 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76f0025-ffb9-425e-ae69-2f1abb5e655e admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:06:04.754760 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76f0025-ffb9-425e-ae69-2f1abb5e655e admin admin] PATH_INFO: `/v3/roles` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:06:04.755433 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76f0025-ffb9-425e-ae69-2f1abb5e655e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:06:04.757344 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76f0025-ffb9-425e-ae69-2f1abb5e655e admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:06:04.758135 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aee9ff8-08f7-492f-aad5-5f6bd156f549 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:06:04.758847 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aee9ff8-08f7-492f-aad5-5f6bd156f549 admin admin] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:06:04.759133 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aee9ff8-08f7-492f-aad5-5f6bd156f549 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:06:04.759405 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aee9ff8-08f7-492f-aad5-5f6bd156f549 admin admin] PATH_INFO: `/v3/users` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:06:04.760045 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aee9ff8-08f7-492f-aad5-5f6bd156f549 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:06:04.762640 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aee9ff8-08f7-492f-aad5-5f6bd156f549 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:06:04.767792 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1363/2716] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 1 15:06:04 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 15:06:04.772654 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76f0025-ffb9-425e-ae69-2f1abb5e655e admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:06:04.784684 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76f0025-ffb9-425e-ae69-2f1abb5e655e admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:06:04.785870 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1354/2717] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Sep 1 15:06:04 2022] POST /identity/v3/users => generated 299 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 15:06:04.789437 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0985a9-a9e1-4a34-863d-6f17c04c2a1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:06:04.789825 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aee9ff8-08f7-492f-aad5-5f6bd156f549 admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:06:04.790166 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0985a9-a9e1-4a34-863d-6f17c04c2a1e admin admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:06:04.790480 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0985a9-a9e1-4a34-863d-6f17c04c2a1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:06:04.790852 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0985a9-a9e1-4a34-863d-6f17c04c2a1e admin admin] PATH_INFO: `/v3/roles` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:06:04.791495 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0985a9-a9e1-4a34-863d-6f17c04c2a1e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:06:04.793631 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0985a9-a9e1-4a34-863d-6f17c04c2a1e admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:06:04.799511 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 15:06:04.800605 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aee9ff8-08f7-492f-aad5-5f6bd156f549 admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:06:04.805747 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6344b93d-cf7f-4d2e-a773-b3bb1d61634f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:06:04.806000 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1364/2718] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 1 15:06:04 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 15:06:04.806444 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6344b93d-cf7f-4d2e-a773-b3bb1d61634f admin admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:06:04.806722 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6344b93d-cf7f-4d2e-a773-b3bb1d61634f admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:06:04.806987 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6344b93d-cf7f-4d2e-a773-b3bb1d61634f admin admin] PATH_INFO: `/v3/roles` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:06:04.808184 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6344b93d-cf7f-4d2e-a773-b3bb1d61634f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:06:04.809970 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6344b93d-cf7f-4d2e-a773-b3bb1d61634f admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:06:04.810173 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0985a9-a9e1-4a34-863d-6f17c04c2a1e admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:06:04.820083 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0985a9-a9e1-4a34-863d-6f17c04c2a1e admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:06:04.821158 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1355/2719] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 1 15:06:04 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 15:06:04.824924 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82a55ab-1aeb-427b-8e43-1558d21feac3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:06:04.825106 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6344b93d-cf7f-4d2e-a773-b3bb1d61634f admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:06:04.825703 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82a55ab-1aeb-427b-8e43-1558d21feac3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:06:04.826051 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82a55ab-1aeb-427b-8e43-1558d21feac3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:06:04.826423 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82a55ab-1aeb-427b-8e43-1558d21feac3 admin admin] PATH_INFO: `/v3/projects/647865839bf544f4a487953c6bac6530/users/b490c67f0e2e49199f6c313025d4b7a6/roles/5d1feef0e13d4874928bd86258c4e3b2` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:06:04.834671 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6344b93d-cf7f-4d2e-a773-b3bb1d61634f admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:06:04.839638 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3517e36b-8004-4948-9176-57cb3b67d36a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:06:04.840303 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3517e36b-8004-4948-9176-57cb3b67d36a admin admin] REQUEST_METHOD: `PUT` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:06:04.840583 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3517e36b-8004-4948-9176-57cb3b67d36a admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:06:04.840853 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3517e36b-8004-4948-9176-57cb3b67d36a admin admin] PATH_INFO: `/v3/projects/67b7e907da6e42a29a017480c2ecd26c/users/e41d78b4ebdd4c239d8e0a6949c38e9a/roles/2a2adb769e55476b82e5014897e0a9c0` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:06:04.841251 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82a55ab-1aeb-427b-8e43-1558d21feac3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=647865839bf544f4a487953c6bac6530, user_id=b490c67f0e2e49199f6c313025d4b7a6, role_id=5d1feef0e13d4874928bd86258c4e3b2)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:06:04.843944 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82a55ab-1aeb-427b-8e43-1558d21feac3 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:06:04.849034 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 15:06:04.858095 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3517e36b-8004-4948-9176-57cb3b67d36a admin admin] RBAC: Authorizing `identity:create_grant(project_id=67b7e907da6e42a29a017480c2ecd26c, user_id=e41d78b4ebdd4c239d8e0a6949c38e9a, role_id=2a2adb769e55476b82e5014897e0a9c0)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:06:04.859515 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1365/2720] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 1 15:06:04 2022] PUT /identity/v3/projects/647865839bf544f4a487953c6bac6530/users/b490c67f0e2e49199f6c313025d4b7a6/roles/5d1feef0e13d4874928bd86258c4e3b2 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 15:06:04.860968 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3517e36b-8004-4948-9176-57cb3b67d36a admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:06:04.871737 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dae28bf-ab7b-4b56-822f-7bf2cfc3d95e None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:06:04.872090 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dae28bf-ab7b-4b56-822f-7bf2cfc3d95e None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:06:04.872431 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dae28bf-ab7b-4b56-822f-7bf2cfc3d95e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:06:04.875226 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1356/2721] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 1 15:06:04 2022] PUT /identity/v3/projects/67b7e907da6e42a29a017480c2ecd26c/users/e41d78b4ebdd4c239d8e0a6949c38e9a/roles/2a2adb769e55476b82e5014897e0a9c0 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 15:06:04.879293 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3517e36b-8004-4948-9176-57cb3b67d36a admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:06:04.900502 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-7dae28bf-ab7b-4b56-822f-7bf2cfc3d95e None None] MFA Rules not processed for user `b490c67f0e2e49199f6c313025d4b7a6`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 15:06:04.901786 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-7dae28bf-ab7b-4b56-822f-7bf2cfc3d95e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:06:04.940163 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3517e36b-8004-4948-9176-57cb3b67d36a admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:06:04.944831 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c942ba4d-8143-46db-9bc9-60dac8848f50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:06:04.945431 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c942ba4d-8143-46db-9bc9-60dac8848f50 admin admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:06:04.945707 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c942ba4d-8143-46db-9bc9-60dac8848f50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:06:04.945998 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c942ba4d-8143-46db-9bc9-60dac8848f50 admin admin] PATH_INFO: `/v3/roles` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:06:04.946696 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c942ba4d-8143-46db-9bc9-60dac8848f50 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:06:04.948293 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c942ba4d-8143-46db-9bc9-60dac8848f50 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:06:04.958214 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1357/2722] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 1 15:06:04 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 15:06:04.962307 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c942ba4d-8143-46db-9bc9-60dac8848f50 admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:06:04.969892 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1366/2723] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 15:06:04 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 15:06:04.972535 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c942ba4d-8143-46db-9bc9-60dac8848f50 admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:06:04.977203 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8ff698-ab98-4613-8cf3-7d1616f3b671 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:06:04.977843 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8ff698-ab98-4613-8cf3-7d1616f3b671 admin admin] REQUEST_METHOD: `PUT` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:06:04.978123 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8ff698-ab98-4613-8cf3-7d1616f3b671 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:06:04.978428 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8ff698-ab98-4613-8cf3-7d1616f3b671 admin admin] PATH_INFO: `/v3/projects/67b7e907da6e42a29a017480c2ecd26c/users/e41d78b4ebdd4c239d8e0a6949c38e9a/roles/2a2adb769e55476b82e5014897e0a9c0` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:06:04.980950 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8ff698-ab98-4613-8cf3-7d1616f3b671 admin admin] RBAC: Authorizing `identity:create_grant(project_id=67b7e907da6e42a29a017480c2ecd26c, user_id=e41d78b4ebdd4c239d8e0a6949c38e9a, role_id=2a2adb769e55476b82e5014897e0a9c0)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:06:04.983258 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dae28bf-ab7b-4b56-822f-7bf2cfc3d95e None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:06:04.983639 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8ff698-ab98-4613-8cf3-7d1616f3b671 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:06:04.984622 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-7dae28bf-ab7b-4b56-822f-7bf2cfc3d95e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:06:04.993982 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1358/2724] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 1 15:06:04 2022] PUT /identity/v3/projects/67b7e907da6e42a29a017480c2ecd26c/users/e41d78b4ebdd4c239d8e0a6949c38e9a/roles/2a2adb769e55476b82e5014897e0a9c0 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 15:06:05.005318 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e471b4e2-b320-4472-8ed5-fc3657b909ad None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:06:05.005608 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e471b4e2-b320-4472-8ed5-fc3657b909ad None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:06:05.005887 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e471b4e2-b320-4472-8ed5-fc3657b909ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:06:05.035194 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-e471b4e2-b320-4472-8ed5-fc3657b909ad None None] MFA Rules not processed for user `e41d78b4ebdd4c239d8e0a6949c38e9a`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 15:06:05.036582 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-e471b4e2-b320-4472-8ed5-fc3657b909ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:06:05.041052 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dae28bf-ab7b-4b56-822f-7bf2cfc3d95e None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:06:05.088923 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56aa7c31-c73d-44ca-90c6-5520e9aaf7b6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3b81739e34e4e648f88b6d73fcb5e46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:06:05.089522 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56aa7c31-c73d-44ca-90c6-5520e9aaf7b6 service cinder] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:06:05.089891 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56aa7c31-c73d-44ca-90c6-5520e9aaf7b6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:06:05.090182 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56aa7c31-c73d-44ca-90c6-5520e9aaf7b6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:06:05.106773 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1359/2725] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 15:06:05 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 15:06:05.117417 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e471b4e2-b320-4472-8ed5-fc3657b909ad None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:06:05.118808 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-e471b4e2-b320-4472-8ed5-fc3657b909ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:06:05.125689 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 15:06:05.131407 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e471b4e2-b320-4472-8ed5-fc3657b909ad None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:06:05.135714 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1213d049-3df5-4563-9148-12cfe81b36d2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3b81739e34e4e648f88b6d73fcb5e46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:06:05.136314 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1213d049-3df5-4563-9148-12cfe81b36d2 service cinder] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:06:05.136593 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1213d049-3df5-4563-9148-12cfe81b36d2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:06:05.136859 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1213d049-3df5-4563-9148-12cfe81b36d2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:06:05.146447 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1213d049-3df5-4563-9148-12cfe81b36d2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:06:05.147906 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1213d049-3df5-4563-9148-12cfe81b36d2 service cinder] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:06:05.149434 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56aa7c31-c73d-44ca-90c6-5520e9aaf7b6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:06:05.151448 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56aa7c31-c73d-44ca-90c6-5520e9aaf7b6 service cinder] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:06:05.158000 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1360/2726] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 15:06:05 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 15:06:05.160833 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1367/2727] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 15:06:04 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 15:06:05.724537 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1213d049-3df5-4563-9148-12cfe81b36d2 service cinder] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:06:05.769137 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 15:06:05.788279 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1213d049-3df5-4563-9148-12cfe81b36d2 service cinder] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:06:05.792831 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611e4c88-d2aa-4f67-b71f-6ed4b079cf7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:06:05.793395 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611e4c88-d2aa-4f67-b71f-6ed4b079cf7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:06:05.793672 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611e4c88-d2aa-4f67-b71f-6ed4b079cf7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:06:05.793942 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611e4c88-d2aa-4f67-b71f-6ed4b079cf7c admin admin] PATH_INFO: `/v3/users/b3bf16245bd74a77928ee9822a4b508d` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:06:05.795330 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611e4c88-d2aa-4f67-b71f-6ed4b079cf7c admin admin] RBAC: Authorizing `identity:delete_user(user_id=b3bf16245bd74a77928ee9822a4b508d)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:06:05.797260 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611e4c88-d2aa-4f67-b71f-6ed4b079cf7c admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:06:05.843388 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-611e4c88-d2aa-4f67-b71f-6ed4b079cf7c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b3bf16245bd74a77928ee9822a4b508d'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:06:05.850165 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-611e4c88-d2aa-4f67-b71f-6ed4b079cf7c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b3bf16245bd74a77928ee9822a4b508d'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:06:05.850808 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-611e4c88-d2aa-4f67-b71f-6ed4b079cf7c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b3bf16245bd74a77928ee9822a4b508d'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:06:05.862813 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-611e4c88-d2aa-4f67-b71f-6ed4b079cf7c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b3bf16245bd74a77928ee9822a4b508d'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:06:05.863436 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-611e4c88-d2aa-4f67-b71f-6ed4b079cf7c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b3bf16245bd74a77928ee9822a4b508d'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:06:05.875378 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1361/2728] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Sep 1 15:06:05 2022] DELETE /identity/v3/users/b3bf16245bd74a77928ee9822a4b508d => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 15:06:05.879939 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9adecf-fb04-4f6e-beb9-567d85cdca18 None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:06:05.880230 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9adecf-fb04-4f6e-beb9-567d85cdca18 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:06:05.880504 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9adecf-fb04-4f6e-beb9-567d85cdca18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:06:05.899672 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-2f9adecf-fb04-4f6e-beb9-567d85cdca18 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 15:06:05.900873 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-2f9adecf-fb04-4f6e-beb9-567d85cdca18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:06:05.961072 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1368/2729] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 15:06:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 15:06:05.971148 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611e4c88-d2aa-4f67-b71f-6ed4b079cf7c admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:06:05.972659 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-611e4c88-d2aa-4f67-b71f-6ed4b079cf7c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:06:06.031244 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611e4c88-d2aa-4f67-b71f-6ed4b079cf7c admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:06:06.035052 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad344930-c151-4e64-866e-015c16b33d2f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2790149639aa418bbc4e4982e93eda44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:06:06.035657 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad344930-c151-4e64-866e-015c16b33d2f service neutron] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:06:06.035937 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad344930-c151-4e64-866e-015c16b33d2f service neutron] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:06:06.036213 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad344930-c151-4e64-866e-015c16b33d2f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:06:06.045945 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad344930-c151-4e64-866e-015c16b33d2f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:06:06.047458 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad344930-c151-4e64-866e-015c16b33d2f service neutron] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:06:06.058370 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1362/2730] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 15:06:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 15:06:06.423106 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9adecf-fb04-4f6e-beb9-567d85cdca18 None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:06:06.424605 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-2f9adecf-fb04-4f6e-beb9-567d85cdca18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:06:06.437503 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9adecf-fb04-4f6e-beb9-567d85cdca18 None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:06:06.442082 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31aa8da0-5336-46fa-8f01-fbc31b3649f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:06:06.442806 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31aa8da0-5336-46fa-8f01-fbc31b3649f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:06:06.443143 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31aa8da0-5336-46fa-8f01-fbc31b3649f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:06:06.443474 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31aa8da0-5336-46fa-8f01-fbc31b3649f0 admin admin] PATH_INFO: `/v3/projects/37c8077b51324140ad47fbdda97f382b` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:06:06.444682 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31aa8da0-5336-46fa-8f01-fbc31b3649f0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=37c8077b51324140ad47fbdda97f382b)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:06:06.446735 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31aa8da0-5336-46fa-8f01-fbc31b3649f0 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:06:06.493332 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-31aa8da0-5336-46fa-8f01-fbc31b3649f0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '37c8077b51324140ad47fbdda97f382b', 'request_id': 'req-31aa8da0-5336-46fa-8f01-fbc31b3649f0'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:06:06.500089 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-31aa8da0-5336-46fa-8f01-fbc31b3649f0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '37c8077b51324140ad47fbdda97f382b', 'request_id': 'req-31aa8da0-5336-46fa-8f01-fbc31b3649f0'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:06:06.506562 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-31aa8da0-5336-46fa-8f01-fbc31b3649f0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '37c8077b51324140ad47fbdda97f382b', 'request_id': 'req-31aa8da0-5336-46fa-8f01-fbc31b3649f0'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:06:06.516573 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-31aa8da0-5336-46fa-8f01-fbc31b3649f0 admin admin] The token cache is being invalidate because project 37c8077b51324140ad47fbdda97f382b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81342) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 15:06:06.516855 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-31aa8da0-5336-46fa-8f01-fbc31b3649f0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:06:06.517471 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-31aa8da0-5336-46fa-8f01-fbc31b3649f0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:06:06.518973 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1369/2731] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Sep 1 15:06:06 2022] DELETE /identity/v3/projects/37c8077b51324140ad47fbdda97f382b => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 15:06:06.522882 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad344930-c151-4e64-866e-015c16b33d2f service neutron] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:06:06.532527 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad344930-c151-4e64-866e-015c16b33d2f service neutron] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:06:06.588345 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a208020-7ff6-4c9c-9335-ea582c042e85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:06:06.588975 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a208020-7ff6-4c9c-9335-ea582c042e85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:06:06.589253 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a208020-7ff6-4c9c-9335-ea582c042e85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:06:06.589522 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a208020-7ff6-4c9c-9335-ea582c042e85 admin admin] PATH_INFO: `/v3/users/e41d78b4ebdd4c239d8e0a6949c38e9a` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:06:06.590706 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a208020-7ff6-4c9c-9335-ea582c042e85 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e41d78b4ebdd4c239d8e0a6949c38e9a)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:06:06.592641 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a208020-7ff6-4c9c-9335-ea582c042e85 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:06:06.597051 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 15:06:06.644979 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-5a208020-7ff6-4c9c-9335-ea582c042e85 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e41d78b4ebdd4c239d8e0a6949c38e9a'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:06:06.657456 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-5a208020-7ff6-4c9c-9335-ea582c042e85 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e41d78b4ebdd4c239d8e0a6949c38e9a'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:06:06.658199 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-5a208020-7ff6-4c9c-9335-ea582c042e85 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e41d78b4ebdd4c239d8e0a6949c38e9a'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:06:06.672254 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-5a208020-7ff6-4c9c-9335-ea582c042e85 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e41d78b4ebdd4c239d8e0a6949c38e9a'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:06:06.673047 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-5a208020-7ff6-4c9c-9335-ea582c042e85 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e41d78b4ebdd4c239d8e0a6949c38e9a'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:06:06.685624 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1363/2732] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Sep 1 15:06:06 2022] DELETE /identity/v3/users/e41d78b4ebdd4c239d8e0a6949c38e9a => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 15:06:06.967416 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31aa8da0-5336-46fa-8f01-fbc31b3649f0 admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:06:06.968855 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-31aa8da0-5336-46fa-8f01-fbc31b3649f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:06:06.983377 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31aa8da0-5336-46fa-8f01-fbc31b3649f0 admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:06:07.040241 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59ceedc-8e68-4104-8972-1ebec1a36e86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:06:07.040868 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59ceedc-8e68-4104-8972-1ebec1a36e86 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:06:07.041143 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59ceedc-8e68-4104-8972-1ebec1a36e86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:06:07.041408 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59ceedc-8e68-4104-8972-1ebec1a36e86 admin admin] PATH_INFO: `/v3/projects/67b7e907da6e42a29a017480c2ecd26c` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:06:07.043184 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59ceedc-8e68-4104-8972-1ebec1a36e86 admin admin] RBAC: Authorizing `identity:delete_project(project_id=67b7e907da6e42a29a017480c2ecd26c)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:06:07.045381 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59ceedc-8e68-4104-8972-1ebec1a36e86 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:06:07.046810 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 15:06:07.091822 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-c59ceedc-8e68-4104-8972-1ebec1a36e86 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '67b7e907da6e42a29a017480c2ecd26c', 'request_id': 'req-c59ceedc-8e68-4104-8972-1ebec1a36e86'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:06:07.097711 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-c59ceedc-8e68-4104-8972-1ebec1a36e86 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '67b7e907da6e42a29a017480c2ecd26c', 'request_id': 'req-c59ceedc-8e68-4104-8972-1ebec1a36e86'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:06:07.104993 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-c59ceedc-8e68-4104-8972-1ebec1a36e86 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '67b7e907da6e42a29a017480c2ecd26c', 'request_id': 'req-c59ceedc-8e68-4104-8972-1ebec1a36e86'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:06:07.115195 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-c59ceedc-8e68-4104-8972-1ebec1a36e86 admin admin] The token cache is being invalidate because project 67b7e907da6e42a29a017480c2ecd26c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81342) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 15:06:07.115477 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-c59ceedc-8e68-4104-8972-1ebec1a36e86 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:06:07.116100 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-c59ceedc-8e68-4104-8972-1ebec1a36e86 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:06:07.117555 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1370/2733] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Sep 1 15:06:06 2022] DELETE /identity/v3/projects/67b7e907da6e42a29a017480c2ecd26c => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 15:06:07.122646 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436f572b-420b-457f-a990-37dac90bd562 None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:06:07.123119 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436f572b-420b-457f-a990-37dac90bd562 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:06:07.123219 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436f572b-420b-457f-a990-37dac90bd562 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:06:07.143004 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-436f572b-420b-457f-a990-37dac90bd562 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 15:06:07.144200 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-436f572b-420b-457f-a990-37dac90bd562 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:06:07.203023 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1364/2734] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 15:06:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 15:06:07.210835 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea416188-fa17-4d26-b8bd-1f54cce5e795 None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:06:07.211117 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea416188-fa17-4d26-b8bd-1f54cce5e795 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:06:07.211386 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea416188-fa17-4d26-b8bd-1f54cce5e795 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:06:07.231150 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-ea416188-fa17-4d26-b8bd-1f54cce5e795 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 15:06:07.232363 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-ea416188-fa17-4d26-b8bd-1f54cce5e795 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:06:07.240094 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1371/2735] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 15:06:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 15:06:07.244378 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436f572b-420b-457f-a990-37dac90bd562 None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:06:07.253992 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436f572b-420b-457f-a990-37dac90bd562 None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:06:07.258626 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c2a2f6-5691-43bc-a21e-c27af7c6127e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:06:07.259239 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c2a2f6-5691-43bc-a21e-c27af7c6127e admin admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:06:07.259517 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c2a2f6-5691-43bc-a21e-c27af7c6127e admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:06:07.259787 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c2a2f6-5691-43bc-a21e-c27af7c6127e admin admin] PATH_INFO: `/v3/domains` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:06:07.260957 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c2a2f6-5691-43bc-a21e-c27af7c6127e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81341) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 15:06:07.261254 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c2a2f6-5691-43bc-a21e-c27af7c6127e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:06:07.262890 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c2a2f6-5691-43bc-a21e-c27af7c6127e admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:06:07.274219 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1365/2736] 10.222.0.33 () {58 vars in 1153 bytes} [Thu Sep 1 15:06:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 15:06:07.279034 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea416188-fa17-4d26-b8bd-1f54cce5e795 None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:06:07.288636 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea416188-fa17-4d26-b8bd-1f54cce5e795 None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:06:07.293302 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e7f07b-92fc-4859-9496-d5b633d9c873 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:06:07.293896 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e7f07b-92fc-4859-9496-d5b633d9c873 admin admin] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:06:07.294174 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e7f07b-92fc-4859-9496-d5b633d9c873 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:06:07.294475 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e7f07b-92fc-4859-9496-d5b633d9c873 admin admin] PATH_INFO: `/v3/projects` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:06:07.295401 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e7f07b-92fc-4859-9496-d5b633d9c873 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:06:07.297309 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e7f07b-92fc-4859-9496-d5b633d9c873 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:06:07.324903 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1372/2737] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Sep 1 15:06:07 2022] POST /identity/v3/projects => generated 385 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 15:06:07.329020 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c2a2f6-5691-43bc-a21e-c27af7c6127e admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:06:07.339635 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c2a2f6-5691-43bc-a21e-c27af7c6127e admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:06:07.393731 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafc54b9-9a23-4e51-8ab0-0feece69a5af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:06:07.394277 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafc54b9-9a23-4e51-8ab0-0feece69a5af admin admin] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:06:07.394555 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafc54b9-9a23-4e51-8ab0-0feece69a5af admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:06:07.394824 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafc54b9-9a23-4e51-8ab0-0feece69a5af admin admin] PATH_INFO: `/v3/users` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:06:07.395436 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafc54b9-9a23-4e51-8ab0-0feece69a5af admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:06:07.397338 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafc54b9-9a23-4e51-8ab0-0feece69a5af admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:06:07.413979 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 15:06:07.419597 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1366/2738] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Sep 1 15:06:07 2022] POST /identity/v3/users => generated 293 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 15:06:07.423938 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e7f07b-92fc-4859-9496-d5b633d9c873 admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:06:07.434666 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e7f07b-92fc-4859-9496-d5b633d9c873 admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:06:07.439292 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e1a4f8-7ace-4955-b886-663ac6d832fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:06:07.439883 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e1a4f8-7ace-4955-b886-663ac6d832fa admin admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:06:07.440263 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e1a4f8-7ace-4955-b886-663ac6d832fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:06:07.440414 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e1a4f8-7ace-4955-b886-663ac6d832fa admin admin] PATH_INFO: `/v3/roles` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:06:07.441111 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e1a4f8-7ace-4955-b886-663ac6d832fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:06:07.442891 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e1a4f8-7ace-4955-b886-663ac6d832fa admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:06:07.456375 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1373/2739] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 1 15:06:07 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 15:06:07.460474 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafc54b9-9a23-4e51-8ab0-0feece69a5af admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:06:07.470323 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafc54b9-9a23-4e51-8ab0-0feece69a5af admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:06:07.475215 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0052bdcc-e46d-4242-b96f-e3c2631a86fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:06:07.475805 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0052bdcc-e46d-4242-b96f-e3c2631a86fb admin admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:06:07.476081 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0052bdcc-e46d-4242-b96f-e3c2631a86fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:06:07.476347 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0052bdcc-e46d-4242-b96f-e3c2631a86fb admin admin] PATH_INFO: `/v3/roles` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:06:07.476983 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0052bdcc-e46d-4242-b96f-e3c2631a86fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:06:07.478607 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0052bdcc-e46d-4242-b96f-e3c2631a86fb admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:06:07.489491 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1367/2740] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 1 15:06:07 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 15:06:07.493599 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e1a4f8-7ace-4955-b886-663ac6d832fa admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:06:07.504043 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e1a4f8-7ace-4955-b886-663ac6d832fa admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:06:07.509534 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522f34da-fea0-41d7-9a17-18da04901b9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:06:07.510197 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522f34da-fea0-41d7-9a17-18da04901b9c admin admin] REQUEST_METHOD: `PUT` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:06:07.511030 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522f34da-fea0-41d7-9a17-18da04901b9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:06:07.511309 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522f34da-fea0-41d7-9a17-18da04901b9c admin admin] PATH_INFO: `/v3/projects/242e4c7bfb054ff0bfe0b77f15309fb7/users/5edb88a1e1a04c3ebf38b6b2771dd522/roles/5d1feef0e13d4874928bd86258c4e3b2` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:06:07.527293 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522f34da-fea0-41d7-9a17-18da04901b9c admin admin] RBAC: Authorizing `identity:create_grant(project_id=242e4c7bfb054ff0bfe0b77f15309fb7, user_id=5edb88a1e1a04c3ebf38b6b2771dd522, role_id=5d1feef0e13d4874928bd86258c4e3b2)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:06:07.529959 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522f34da-fea0-41d7-9a17-18da04901b9c admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:06:07.543696 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1374/2741] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 1 15:06:07 2022] PUT /identity/v3/projects/242e4c7bfb054ff0bfe0b77f15309fb7/users/5edb88a1e1a04c3ebf38b6b2771dd522/roles/5d1feef0e13d4874928bd86258c4e3b2 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 15:06:07.615310 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7140198-1435-42a6-ba0e-bb98d9ef7d6b None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:06:07.615594 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7140198-1435-42a6-ba0e-bb98d9ef7d6b None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:06:07.615866 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7140198-1435-42a6-ba0e-bb98d9ef7d6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:06:07.644210 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-e7140198-1435-42a6-ba0e-bb98d9ef7d6b None None] MFA Rules not processed for user `5edb88a1e1a04c3ebf38b6b2771dd522`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 15:06:07.645354 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-e7140198-1435-42a6-ba0e-bb98d9ef7d6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:06:07.708838 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1368/2742] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 15:06:07 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 15:06:07.719825 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522f34da-fea0-41d7-9a17-18da04901b9c admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:06:07.721509 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-522f34da-fea0-41d7-9a17-18da04901b9c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:06:07.778540 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522f34da-fea0-41d7-9a17-18da04901b9c admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:06:07.782171 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5cefa3-a674-411c-bb28-c83f52400022 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3b81739e34e4e648f88b6d73fcb5e46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:06:07.782801 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5cefa3-a674-411c-bb28-c83f52400022 service cinder] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:06:07.783083 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5cefa3-a674-411c-bb28-c83f52400022 service cinder] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:06:07.783360 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5cefa3-a674-411c-bb28-c83f52400022 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:06:07.792020 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5cefa3-a674-411c-bb28-c83f52400022 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:06:07.793447 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5cefa3-a674-411c-bb28-c83f52400022 service cinder] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:06:07.802430 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1375/2743] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 15:06:07 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 15:06:14.337163 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7140198-1435-42a6-ba0e-bb98d9ef7d6b None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:06:14.338610 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-e7140198-1435-42a6-ba0e-bb98d9ef7d6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:06:14.408652 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7140198-1435-42a6-ba0e-bb98d9ef7d6b None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:06:14.413463 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25c6039-3b3f-4539-ad33-c176a1cd0b42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:06:14.414093 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25c6039-3b3f-4539-ad33-c176a1cd0b42 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:06:14.414404 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25c6039-3b3f-4539-ad33-c176a1cd0b42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:06:14.414680 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25c6039-3b3f-4539-ad33-c176a1cd0b42 admin admin] PATH_INFO: `/v3/users/b490c67f0e2e49199f6c313025d4b7a6` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:06:14.415805 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25c6039-3b3f-4539-ad33-c176a1cd0b42 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b490c67f0e2e49199f6c313025d4b7a6)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:06:14.418386 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25c6039-3b3f-4539-ad33-c176a1cd0b42 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:06:14.465935 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-b25c6039-3b3f-4539-ad33-c176a1cd0b42 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b490c67f0e2e49199f6c313025d4b7a6'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:06:14.473493 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-b25c6039-3b3f-4539-ad33-c176a1cd0b42 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b490c67f0e2e49199f6c313025d4b7a6'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:06:14.474229 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-b25c6039-3b3f-4539-ad33-c176a1cd0b42 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b490c67f0e2e49199f6c313025d4b7a6'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:06:14.489220 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-b25c6039-3b3f-4539-ad33-c176a1cd0b42 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b490c67f0e2e49199f6c313025d4b7a6'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:06:14.490268 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-b25c6039-3b3f-4539-ad33-c176a1cd0b42 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b490c67f0e2e49199f6c313025d4b7a6'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:06:14.500037 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 15:06:14.506670 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1369/2744] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Sep 1 15:06:14 2022] DELETE /identity/v3/users/b490c67f0e2e49199f6c313025d4b7a6 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 15:06:14.513300 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab841e2-84dd-4284-a487-89c8322d279b None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:06:14.513776 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab841e2-84dd-4284-a487-89c8322d279b None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:06:14.513776 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab841e2-84dd-4284-a487-89c8322d279b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:06:14.535282 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-1ab841e2-84dd-4284-a487-89c8322d279b None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 15:06:14.536501 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-1ab841e2-84dd-4284-a487-89c8322d279b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:06:14.560944 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 15:06:14.599081 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1376/2745] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 15:06:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 15:06:14.609476 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25c6039-3b3f-4539-ad33-c176a1cd0b42 admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:06:14.611001 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-b25c6039-3b3f-4539-ad33-c176a1cd0b42 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:06:14.672703 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25c6039-3b3f-4539-ad33-c176a1cd0b42 admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:06:14.676457 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932e67ba-32db-4c0f-827f-5c4dbbc939d2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2790149639aa418bbc4e4982e93eda44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:06:14.677055 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932e67ba-32db-4c0f-827f-5c4dbbc939d2 service neutron] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:06:14.677333 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932e67ba-32db-4c0f-827f-5c4dbbc939d2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:06:14.677601 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932e67ba-32db-4c0f-827f-5c4dbbc939d2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:06:14.686737 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932e67ba-32db-4c0f-827f-5c4dbbc939d2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:06:14.688157 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932e67ba-32db-4c0f-827f-5c4dbbc939d2 service neutron] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:06:14.697834 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1370/2746] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 15:06:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 15:06:14.986830 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab841e2-84dd-4284-a487-89c8322d279b None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:06:14.988692 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-1ab841e2-84dd-4284-a487-89c8322d279b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:06:15.003920 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab841e2-84dd-4284-a487-89c8322d279b None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:06:15.009794 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c86a50-0c7d-4d44-9dc0-b1a2cdd2bf68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:06:15.010531 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c86a50-0c7d-4d44-9dc0-b1a2cdd2bf68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:06:15.011617 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c86a50-0c7d-4d44-9dc0-b1a2cdd2bf68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:06:15.011959 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c86a50-0c7d-4d44-9dc0-b1a2cdd2bf68 admin admin] PATH_INFO: `/v3/projects/647865839bf544f4a487953c6bac6530` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:06:15.013422 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c86a50-0c7d-4d44-9dc0-b1a2cdd2bf68 admin admin] RBAC: Authorizing `identity:delete_project(project_id=647865839bf544f4a487953c6bac6530)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:06:15.015674 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c86a50-0c7d-4d44-9dc0-b1a2cdd2bf68 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:06:15.064856 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-b3c86a50-0c7d-4d44-9dc0-b1a2cdd2bf68 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '647865839bf544f4a487953c6bac6530', 'request_id': 'req-b3c86a50-0c7d-4d44-9dc0-b1a2cdd2bf68'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:06:15.071030 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-b3c86a50-0c7d-4d44-9dc0-b1a2cdd2bf68 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '647865839bf544f4a487953c6bac6530', 'request_id': 'req-b3c86a50-0c7d-4d44-9dc0-b1a2cdd2bf68'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:06:15.082222 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-b3c86a50-0c7d-4d44-9dc0-b1a2cdd2bf68 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '647865839bf544f4a487953c6bac6530', 'request_id': 'req-b3c86a50-0c7d-4d44-9dc0-b1a2cdd2bf68'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:06:15.094834 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-b3c86a50-0c7d-4d44-9dc0-b1a2cdd2bf68 admin admin] The token cache is being invalidate because project 647865839bf544f4a487953c6bac6530 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81342) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 15:06:15.095121 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-b3c86a50-0c7d-4d44-9dc0-b1a2cdd2bf68 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:06:15.095771 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-b3c86a50-0c7d-4d44-9dc0-b1a2cdd2bf68 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:06:15.097564 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1377/2747] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Sep 1 15:06:14 2022] DELETE /identity/v3/projects/647865839bf544f4a487953c6bac6530 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 15:06:15.102514 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07f36b4-4fa0-4de2-bc29-1489cd4a695f None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:06:15.102781 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07f36b4-4fa0-4de2-bc29-1489cd4a695f None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:06:15.103060 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07f36b4-4fa0-4de2-bc29-1489cd4a695f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:06:15.123702 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-e07f36b4-4fa0-4de2-bc29-1489cd4a695f None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 15:06:15.124986 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-e07f36b4-4fa0-4de2-bc29-1489cd4a695f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:06:15.213077 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1371/2748] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 15:06:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 15:06:15.224070 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac92b50-c95c-47e4-8c37-32ff0b85c2f9 None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:06:15.224991 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac92b50-c95c-47e4-8c37-32ff0b85c2f9 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:06:15.224991 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac92b50-c95c-47e4-8c37-32ff0b85c2f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:06:15.246626 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-dac92b50-c95c-47e4-8c37-32ff0b85c2f9 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 15:06:15.248061 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-dac92b50-c95c-47e4-8c37-32ff0b85c2f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:06:15.256943 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1378/2749] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 15:06:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 15:06:15.260802 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07f36b4-4fa0-4de2-bc29-1489cd4a695f None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:06:15.273630 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07f36b4-4fa0-4de2-bc29-1489cd4a695f None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:06:15.279219 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98df86ab-b510-4445-8fe3-36a5c01cf97d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:06:15.279554 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98df86ab-b510-4445-8fe3-36a5c01cf97d admin admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:06:15.279835 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98df86ab-b510-4445-8fe3-36a5c01cf97d admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:06:15.280160 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98df86ab-b510-4445-8fe3-36a5c01cf97d admin admin] PATH_INFO: `/v3/domains` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:06:15.280941 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98df86ab-b510-4445-8fe3-36a5c01cf97d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81341) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 15:06:15.281248 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98df86ab-b510-4445-8fe3-36a5c01cf97d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:06:15.283060 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98df86ab-b510-4445-8fe3-36a5c01cf97d admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:06:15.303224 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1372/2750] 10.222.0.33 () {58 vars in 1153 bytes} [Thu Sep 1 15:06:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 15:06:15.304385 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac92b50-c95c-47e4-8c37-32ff0b85c2f9 None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:06:15.314299 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac92b50-c95c-47e4-8c37-32ff0b85c2f9 None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:06:15.318989 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa26a6bb-9241-49e1-abc3-05a4fbfac6d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:06:15.319579 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa26a6bb-9241-49e1-abc3-05a4fbfac6d1 admin admin] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:06:15.319856 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa26a6bb-9241-49e1-abc3-05a4fbfac6d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:06:15.320122 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa26a6bb-9241-49e1-abc3-05a4fbfac6d1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:06:15.320768 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa26a6bb-9241-49e1-abc3-05a4fbfac6d1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:06:15.322968 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa26a6bb-9241-49e1-abc3-05a4fbfac6d1 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:06:15.356659 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1379/2751] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Sep 1 15:06:15 2022] POST /identity/v3/projects => generated 383 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 15:06:15.360756 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98df86ab-b510-4445-8fe3-36a5c01cf97d admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:06:15.371623 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98df86ab-b510-4445-8fe3-36a5c01cf97d admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:06:15.428936 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63305a7-1237-46f0-8757-eecc36901f2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:06:15.429530 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63305a7-1237-46f0-8757-eecc36901f2d admin admin] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:06:15.429806 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63305a7-1237-46f0-8757-eecc36901f2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:06:15.430124 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63305a7-1237-46f0-8757-eecc36901f2d admin admin] PATH_INFO: `/v3/users` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:06:15.430787 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63305a7-1237-46f0-8757-eecc36901f2d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:06:15.432750 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63305a7-1237-46f0-8757-eecc36901f2d admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:06:15.453491 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1373/2752] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Sep 1 15:06:15 2022] POST /identity/v3/users => generated 292 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 15:06:15.457540 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa26a6bb-9241-49e1-abc3-05a4fbfac6d1 admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:06:15.468387 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa26a6bb-9241-49e1-abc3-05a4fbfac6d1 admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:06:15.473068 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b164bc8f-fd22-48d1-a372-7ba58ed359a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:06:15.473659 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b164bc8f-fd22-48d1-a372-7ba58ed359a5 admin admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:06:15.473932 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b164bc8f-fd22-48d1-a372-7ba58ed359a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:06:15.474197 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b164bc8f-fd22-48d1-a372-7ba58ed359a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:06:15.474864 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b164bc8f-fd22-48d1-a372-7ba58ed359a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:06:15.476450 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b164bc8f-fd22-48d1-a372-7ba58ed359a5 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:06:15.486925 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1380/2753] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 1 15:06:15 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 15:06:15.490785 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63305a7-1237-46f0-8757-eecc36901f2d admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:06:15.501167 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63305a7-1237-46f0-8757-eecc36901f2d admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:06:15.505790 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f986707b-ce65-4153-ad9a-b94f381fdd59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:06:15.506432 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f986707b-ce65-4153-ad9a-b94f381fdd59 admin admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:06:15.506710 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f986707b-ce65-4153-ad9a-b94f381fdd59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:06:15.507000 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f986707b-ce65-4153-ad9a-b94f381fdd59 admin admin] PATH_INFO: `/v3/roles` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:06:15.507626 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f986707b-ce65-4153-ad9a-b94f381fdd59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:06:15.509258 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f986707b-ce65-4153-ad9a-b94f381fdd59 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:06:15.519488 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1374/2754] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 1 15:06:15 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 15:06:15.523338 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b164bc8f-fd22-48d1-a372-7ba58ed359a5 admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:06:15.533087 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b164bc8f-fd22-48d1-a372-7ba58ed359a5 admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:06:15.537585 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c1c5e6-84e0-462f-b6d8-e99891cda2cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:06:15.538216 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c1c5e6-84e0-462f-b6d8-e99891cda2cc admin admin] REQUEST_METHOD: `PUT` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:06:15.538529 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c1c5e6-84e0-462f-b6d8-e99891cda2cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:06:15.538801 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c1c5e6-84e0-462f-b6d8-e99891cda2cc admin admin] PATH_INFO: `/v3/projects/a7311840457040a992b3bebaa2d32843/users/70cd6c18012049419d7fa7b591b4bd28/roles/5d1feef0e13d4874928bd86258c4e3b2` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:06:15.553657 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c1c5e6-84e0-462f-b6d8-e99891cda2cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=a7311840457040a992b3bebaa2d32843, user_id=70cd6c18012049419d7fa7b591b4bd28, role_id=5d1feef0e13d4874928bd86258c4e3b2)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:06:15.556300 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c1c5e6-84e0-462f-b6d8-e99891cda2cc admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:06:15.569472 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1381/2755] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 1 15:06:15 2022] PUT /identity/v3/projects/a7311840457040a992b3bebaa2d32843/users/70cd6c18012049419d7fa7b591b4bd28/roles/5d1feef0e13d4874928bd86258c4e3b2 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 15:06:15.574561 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1a65c2-8418-4a78-a6c5-5e69b210e2e4 None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:06:15.574847 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1a65c2-8418-4a78-a6c5-5e69b210e2e4 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:06:15.575120 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1a65c2-8418-4a78-a6c5-5e69b210e2e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:06:15.588162 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 15:06:15.594499 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-2a1a65c2-8418-4a78-a6c5-5e69b210e2e4 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 15:06:15.595696 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-2a1a65c2-8418-4a78-a6c5-5e69b210e2e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:06:15.656298 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1375/2756] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 15:06:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 15:06:15.668918 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c1c5e6-84e0-462f-b6d8-e99891cda2cc admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:06:15.670938 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-27c1c5e6-84e0-462f-b6d8-e99891cda2cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:06:15.723810 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 15:06:15.728072 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c1c5e6-84e0-462f-b6d8-e99891cda2cc admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:06:15.733453 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de15d3d8-edb9-4705-a91d-be113a94bc60 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2790149639aa418bbc4e4982e93eda44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:06:15.734461 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de15d3d8-edb9-4705-a91d-be113a94bc60 service neutron] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:06:15.734881 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de15d3d8-edb9-4705-a91d-be113a94bc60 service neutron] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:06:15.735202 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de15d3d8-edb9-4705-a91d-be113a94bc60 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:06:15.744838 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de15d3d8-edb9-4705-a91d-be113a94bc60 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:06:15.746257 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de15d3d8-edb9-4705-a91d-be113a94bc60 service neutron] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:06:15.755837 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1382/2757] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 15:06:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 15:06:16.244110 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1a65c2-8418-4a78-a6c5-5e69b210e2e4 None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:06:16.245609 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-2a1a65c2-8418-4a78-a6c5-5e69b210e2e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:06:16.257919 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1a65c2-8418-4a78-a6c5-5e69b210e2e4 None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:06:16.263101 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3245391-adb0-46df-a52a-575afa84b27f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:06:16.263841 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3245391-adb0-46df-a52a-575afa84b27f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:06:16.264125 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3245391-adb0-46df-a52a-575afa84b27f admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:06:16.264397 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3245391-adb0-46df-a52a-575afa84b27f admin admin] PATH_INFO: `/v3/users/5edb88a1e1a04c3ebf38b6b2771dd522` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:06:16.265515 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3245391-adb0-46df-a52a-575afa84b27f admin admin] RBAC: Authorizing `identity:delete_user(user_id=5edb88a1e1a04c3ebf38b6b2771dd522)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:06:16.267601 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3245391-adb0-46df-a52a-575afa84b27f admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:06:16.327869 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-f3245391-adb0-46df-a52a-575afa84b27f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5edb88a1e1a04c3ebf38b6b2771dd522'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:06:16.338173 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-f3245391-adb0-46df-a52a-575afa84b27f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5edb88a1e1a04c3ebf38b6b2771dd522'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:06:16.338833 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-f3245391-adb0-46df-a52a-575afa84b27f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5edb88a1e1a04c3ebf38b6b2771dd522'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:06:16.356360 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-f3245391-adb0-46df-a52a-575afa84b27f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5edb88a1e1a04c3ebf38b6b2771dd522'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:06:16.356360 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-f3245391-adb0-46df-a52a-575afa84b27f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5edb88a1e1a04c3ebf38b6b2771dd522'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:06:16.370690 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1376/2758] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Sep 1 15:06:16 2022] DELETE /identity/v3/users/5edb88a1e1a04c3ebf38b6b2771dd522 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 15:06:16.376444 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a204e8b-b353-4047-a74c-c6b631b9fa1f None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:06:16.376658 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a204e8b-b353-4047-a74c-c6b631b9fa1f None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:06:16.376912 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a204e8b-b353-4047-a74c-c6b631b9fa1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:06:16.406489 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-0a204e8b-b353-4047-a74c-c6b631b9fa1f None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 15:06:16.407798 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-0a204e8b-b353-4047-a74c-c6b631b9fa1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:06:16.509919 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1383/2759] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 15:06:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 15:06:16.518025 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3245391-adb0-46df-a52a-575afa84b27f admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:06:16.519825 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-f3245391-adb0-46df-a52a-575afa84b27f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:06:16.623379 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3245391-adb0-46df-a52a-575afa84b27f admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:06:16.627978 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5afaa00-cd86-4bce-81a5-1b70fc1377e0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2790149639aa418bbc4e4982e93eda44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:06:16.628763 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5afaa00-cd86-4bce-81a5-1b70fc1377e0 service neutron] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:06:16.629090 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5afaa00-cd86-4bce-81a5-1b70fc1377e0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:06:16.629431 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5afaa00-cd86-4bce-81a5-1b70fc1377e0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:06:16.640911 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5afaa00-cd86-4bce-81a5-1b70fc1377e0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:06:16.642634 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5afaa00-cd86-4bce-81a5-1b70fc1377e0 service neutron] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:06:16.654562 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1377/2760] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 15:06:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 15:06:17.077490 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a204e8b-b353-4047-a74c-c6b631b9fa1f None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:06:17.079382 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-0a204e8b-b353-4047-a74c-c6b631b9fa1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:06:17.105493 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a204e8b-b353-4047-a74c-c6b631b9fa1f None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:06:17.112211 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdfa96c-291b-45cd-a8fa-287cd7184596 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:06:17.113153 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdfa96c-291b-45cd-a8fa-287cd7184596 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:06:17.114029 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdfa96c-291b-45cd-a8fa-287cd7184596 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:06:17.114029 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdfa96c-291b-45cd-a8fa-287cd7184596 admin admin] PATH_INFO: `/v3/projects/242e4c7bfb054ff0bfe0b77f15309fb7` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:06:17.116027 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfdfa96c-291b-45cd-a8fa-287cd7184596 admin admin] RBAC: Authorizing `identity:delete_project(project_id=242e4c7bfb054ff0bfe0b77f15309fb7)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:06:17.119781 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfdfa96c-291b-45cd-a8fa-287cd7184596 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:06:17.201511 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-bfdfa96c-291b-45cd-a8fa-287cd7184596 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '242e4c7bfb054ff0bfe0b77f15309fb7', 'request_id': 'req-bfdfa96c-291b-45cd-a8fa-287cd7184596'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:06:17.209355 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-bfdfa96c-291b-45cd-a8fa-287cd7184596 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '242e4c7bfb054ff0bfe0b77f15309fb7', 'request_id': 'req-bfdfa96c-291b-45cd-a8fa-287cd7184596'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:06:17.218358 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-bfdfa96c-291b-45cd-a8fa-287cd7184596 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '242e4c7bfb054ff0bfe0b77f15309fb7', 'request_id': 'req-bfdfa96c-291b-45cd-a8fa-287cd7184596'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:06:17.230921 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-bfdfa96c-291b-45cd-a8fa-287cd7184596 admin admin] The token cache is being invalidate because project 242e4c7bfb054ff0bfe0b77f15309fb7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81342) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 15:06:17.231208 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-bfdfa96c-291b-45cd-a8fa-287cd7184596 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:06:17.232071 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-bfdfa96c-291b-45cd-a8fa-287cd7184596 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:06:17.233658 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1384/2761] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Sep 1 15:06:17 2022] DELETE /identity/v3/projects/242e4c7bfb054ff0bfe0b77f15309fb7 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 15:06:17.240119 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a681dc-73e0-4e08-8c22-883d49e57cda None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:06:17.240579 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a681dc-73e0-4e08-8c22-883d49e57cda None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:06:17.240925 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a681dc-73e0-4e08-8c22-883d49e57cda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:06:17.262545 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-49a681dc-73e0-4e08-8c22-883d49e57cda None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 15:06:17.264179 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-49a681dc-73e0-4e08-8c22-883d49e57cda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:06:17.320623 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 15:06:17.386639 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1378/2762] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 15:06:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 15:06:17.394470 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85fe259-8141-4a4d-988f-2e69551723c4 None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:06:17.394761 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85fe259-8141-4a4d-988f-2e69551723c4 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:06:17.395037 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85fe259-8141-4a4d-988f-2e69551723c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:06:17.416433 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-a85fe259-8141-4a4d-988f-2e69551723c4 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 15:06:17.418647 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-a85fe259-8141-4a4d-988f-2e69551723c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:06:17.431847 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1385/2763] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 15:06:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 15:06:17.436092 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a681dc-73e0-4e08-8c22-883d49e57cda None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:06:17.446987 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a681dc-73e0-4e08-8c22-883d49e57cda None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:06:17.451867 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4507c2-1388-4b28-bec9-f1f1b6853f2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:06:17.452511 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4507c2-1388-4b28-bec9-f1f1b6853f2a admin admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:06:17.452792 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4507c2-1388-4b28-bec9-f1f1b6853f2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:06:17.453253 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4507c2-1388-4b28-bec9-f1f1b6853f2a admin admin] PATH_INFO: `/v3/domains` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:06:17.453947 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4507c2-1388-4b28-bec9-f1f1b6853f2a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81341) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 15:06:17.454273 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4507c2-1388-4b28-bec9-f1f1b6853f2a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:06:17.455909 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4507c2-1388-4b28-bec9-f1f1b6853f2a admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:06:17.472296 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1379/2764] 10.222.0.33 () {58 vars in 1153 bytes} [Thu Sep 1 15:06:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 15:06:17.473528 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85fe259-8141-4a4d-988f-2e69551723c4 None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:06:17.483776 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85fe259-8141-4a4d-988f-2e69551723c4 None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:06:17.490300 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2015c32a-75c2-4c5b-a404-a50d38704360 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:06:17.491013 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2015c32a-75c2-4c5b-a404-a50d38704360 admin admin] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:06:17.491290 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2015c32a-75c2-4c5b-a404-a50d38704360 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:06:17.491558 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2015c32a-75c2-4c5b-a404-a50d38704360 admin admin] PATH_INFO: `/v3/projects` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:06:17.492191 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2015c32a-75c2-4c5b-a404-a50d38704360 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:06:17.494213 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2015c32a-75c2-4c5b-a404-a50d38704360 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:06:17.521778 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1386/2765] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Sep 1 15:06:17 2022] POST /identity/v3/projects => generated 383 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 15:06:17.526033 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4507c2-1388-4b28-bec9-f1f1b6853f2a admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:06:17.536975 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4507c2-1388-4b28-bec9-f1f1b6853f2a admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:06:17.599714 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a2d699-1354-4e07-a09b-069c2370d4f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:06:17.600411 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a2d699-1354-4e07-a09b-069c2370d4f1 admin admin] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:06:17.600755 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a2d699-1354-4e07-a09b-069c2370d4f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:06:17.601107 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a2d699-1354-4e07-a09b-069c2370d4f1 admin admin] PATH_INFO: `/v3/users` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:06:17.601807 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a2d699-1354-4e07-a09b-069c2370d4f1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:06:17.604004 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a2d699-1354-4e07-a09b-069c2370d4f1 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:06:17.624675 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1380/2766] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Sep 1 15:06:17 2022] POST /identity/v3/users => generated 292 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 15:06:17.628781 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2015c32a-75c2-4c5b-a404-a50d38704360 admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:06:17.639456 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2015c32a-75c2-4c5b-a404-a50d38704360 admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:06:17.644103 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad8b252-abf4-4ed8-bba0-56a13a728b21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:06:17.644768 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad8b252-abf4-4ed8-bba0-56a13a728b21 admin admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:06:17.645107 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad8b252-abf4-4ed8-bba0-56a13a728b21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:06:17.645473 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad8b252-abf4-4ed8-bba0-56a13a728b21 admin admin] PATH_INFO: `/v3/roles` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:06:17.646177 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad8b252-abf4-4ed8-bba0-56a13a728b21 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:06:17.647859 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad8b252-abf4-4ed8-bba0-56a13a728b21 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:06:17.673055 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1387/2767] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 1 15:06:17 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 15:06:17.677046 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a2d699-1354-4e07-a09b-069c2370d4f1 admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:06:17.687218 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a2d699-1354-4e07-a09b-069c2370d4f1 admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:06:17.691986 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe74f1d4-8f72-45ce-aa7f-025bbe00b362 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:06:17.692590 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe74f1d4-8f72-45ce-aa7f-025bbe00b362 admin admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:06:17.692873 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe74f1d4-8f72-45ce-aa7f-025bbe00b362 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:06:17.693148 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe74f1d4-8f72-45ce-aa7f-025bbe00b362 admin admin] PATH_INFO: `/v3/roles` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:06:17.693799 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe74f1d4-8f72-45ce-aa7f-025bbe00b362 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:06:17.695614 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe74f1d4-8f72-45ce-aa7f-025bbe00b362 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:06:17.708343 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1381/2768] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 1 15:06:17 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 15:06:17.712720 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad8b252-abf4-4ed8-bba0-56a13a728b21 admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:06:17.724003 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad8b252-abf4-4ed8-bba0-56a13a728b21 admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:06:17.729640 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ae2f50-d5c8-47ce-a6b1-0caae8fa4bf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:06:17.730006 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ae2f50-d5c8-47ce-a6b1-0caae8fa4bf5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:06:17.730369 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ae2f50-d5c8-47ce-a6b1-0caae8fa4bf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:06:17.730579 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ae2f50-d5c8-47ce-a6b1-0caae8fa4bf5 admin admin] PATH_INFO: `/v3/projects/0669894d406746cbb39ad80d2c5548bd/users/6aeaa01b99064b65873c950a86dc68c5/roles/5d1feef0e13d4874928bd86258c4e3b2` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:06:17.747528 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ae2f50-d5c8-47ce-a6b1-0caae8fa4bf5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0669894d406746cbb39ad80d2c5548bd, user_id=6aeaa01b99064b65873c950a86dc68c5, role_id=5d1feef0e13d4874928bd86258c4e3b2)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:06:17.750203 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ae2f50-d5c8-47ce-a6b1-0caae8fa4bf5 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:06:17.766055 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1388/2769] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 1 15:06:17 2022] PUT /identity/v3/projects/0669894d406746cbb39ad80d2c5548bd/users/6aeaa01b99064b65873c950a86dc68c5/roles/5d1feef0e13d4874928bd86258c4e3b2 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 15:06:17.778565 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797d4ae0-f316-4e48-b471-a7a389fe2370 None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:06:17.778790 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797d4ae0-f316-4e48-b471-a7a389fe2370 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:06:17.779015 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797d4ae0-f316-4e48-b471-a7a389fe2370 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:06:17.955892 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-797d4ae0-f316-4e48-b471-a7a389fe2370 None None] MFA Rules not processed for user `6aeaa01b99064b65873c950a86dc68c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 15:06:17.957425 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-797d4ae0-f316-4e48-b471-a7a389fe2370 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:06:18.047386 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1382/2770] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 15:06:17 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 270 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 15:06:18.060139 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ae2f50-d5c8-47ce-a6b1-0caae8fa4bf5 admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:06:18.062685 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-f4ae2f50-d5c8-47ce-a6b1-0caae8fa4bf5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:06:18.104025 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 15:06:18.205860 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ae2f50-d5c8-47ce-a6b1-0caae8fa4bf5 admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:06:18.211506 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a26edc5-7abe-4b92-ac29-64b4e1304932 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3b81739e34e4e648f88b6d73fcb5e46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:06:18.212876 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a26edc5-7abe-4b92-ac29-64b4e1304932 service cinder] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:06:18.213337 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a26edc5-7abe-4b92-ac29-64b4e1304932 service cinder] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:06:18.213778 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a26edc5-7abe-4b92-ac29-64b4e1304932 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:06:18.227191 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a26edc5-7abe-4b92-ac29-64b4e1304932 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:06:18.229714 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a26edc5-7abe-4b92-ac29-64b4e1304932 service cinder] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:06:18.244260 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1389/2771] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 15:06:18 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 15:06:21.022955 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797d4ae0-f316-4e48-b471-a7a389fe2370 None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:06:21.060299 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 15:06:21.160956 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797d4ae0-f316-4e48-b471-a7a389fe2370 None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:06:21.175948 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1eebb9b-70a4-418c-ba95-67541b546a79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:06:21.176620 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1eebb9b-70a4-418c-ba95-67541b546a79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:06:21.177233 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1eebb9b-70a4-418c-ba95-67541b546a79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:06:21.177361 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1eebb9b-70a4-418c-ba95-67541b546a79 admin admin] PATH_INFO: `/v3/users/6aeaa01b99064b65873c950a86dc68c5` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:06:21.182089 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1eebb9b-70a4-418c-ba95-67541b546a79 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6aeaa01b99064b65873c950a86dc68c5)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:06:21.184995 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1eebb9b-70a4-418c-ba95-67541b546a79 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:06:21.278425 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-b1eebb9b-70a4-418c-ba95-67541b546a79 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6aeaa01b99064b65873c950a86dc68c5'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:06:21.298885 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-b1eebb9b-70a4-418c-ba95-67541b546a79 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6aeaa01b99064b65873c950a86dc68c5'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:06:21.299986 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-b1eebb9b-70a4-418c-ba95-67541b546a79 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6aeaa01b99064b65873c950a86dc68c5'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:06:21.331240 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-b1eebb9b-70a4-418c-ba95-67541b546a79 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6aeaa01b99064b65873c950a86dc68c5'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:06:21.335177 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-b1eebb9b-70a4-418c-ba95-67541b546a79 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6aeaa01b99064b65873c950a86dc68c5'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:06:21.356728 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1383/2772] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Sep 1 15:06:21 2022] DELETE /identity/v3/users/6aeaa01b99064b65873c950a86dc68c5 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 15:06:21.361147 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10cbdd3-1454-4d06-b323-1d2c1ac702ac None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:06:21.361650 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10cbdd3-1454-4d06-b323-1d2c1ac702ac None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:06:21.361750 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10cbdd3-1454-4d06-b323-1d2c1ac702ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:06:21.388068 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-d10cbdd3-1454-4d06-b323-1d2c1ac702ac None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 15:06:21.389313 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-d10cbdd3-1454-4d06-b323-1d2c1ac702ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:06:21.408052 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4478080-e0f9-4746-b222-00a0904f322a None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:06:21.408052 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4478080-e0f9-4746-b222-00a0904f322a None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:06:21.408052 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4478080-e0f9-4746-b222-00a0904f322a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:06:21.451021 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-e4478080-e0f9-4746-b222-00a0904f322a None None] MFA Rules not processed for user `70cd6c18012049419d7fa7b591b4bd28`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 15:06:21.452256 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-e4478080-e0f9-4746-b222-00a0904f322a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:06:21.479014 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1390/2773] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 15:06:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 15:06:21.485439 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10cbdd3-1454-4d06-b323-1d2c1ac702ac None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:06:21.486815 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-d10cbdd3-1454-4d06-b323-1d2c1ac702ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:06:21.521021 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1384/2774] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 15:06:21 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 15:06:21.525368 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4478080-e0f9-4746-b222-00a0904f322a None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:06:21.527353 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-e4478080-e0f9-4746-b222-00a0904f322a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:06:21.540591 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4478080-e0f9-4746-b222-00a0904f322a None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:06:21.545451 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5612fd53-50d2-4414-a7ab-ea27cb35de89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:06:21.545652 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10cbdd3-1454-4d06-b323-1d2c1ac702ac None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:06:21.546040 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5612fd53-50d2-4414-a7ab-ea27cb35de89 admin admin] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:06:21.546424 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5612fd53-50d2-4414-a7ab-ea27cb35de89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:06:21.546693 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5612fd53-50d2-4414-a7ab-ea27cb35de89 admin admin] PATH_INFO: `/v3/projects` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:06:21.547324 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5612fd53-50d2-4414-a7ab-ea27cb35de89 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:06:21.549228 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5612fd53-50d2-4414-a7ab-ea27cb35de89 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:06:21.553185 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac8e1f2-2e90-4e0a-8e45-b1c25beed332 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2790149639aa418bbc4e4982e93eda44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:06:21.553873 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac8e1f2-2e90-4e0a-8e45-b1c25beed332 service neutron] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:06:21.554153 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac8e1f2-2e90-4e0a-8e45-b1c25beed332 service neutron] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:06:21.554460 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac8e1f2-2e90-4e0a-8e45-b1c25beed332 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:06:21.564382 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac8e1f2-2e90-4e0a-8e45-b1c25beed332 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:06:21.565856 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac8e1f2-2e90-4e0a-8e45-b1c25beed332 service neutron] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:06:21.576696 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1391/2775] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 15:06:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 15:06:21.579731 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1385/2776] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Sep 1 15:06:21 2022] POST /identity/v3/projects => generated 385 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 15:06:21.583430 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac8e1f2-2e90-4e0a-8e45-b1c25beed332 service neutron] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:06:21.593411 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac8e1f2-2e90-4e0a-8e45-b1c25beed332 service neutron] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:06:21.650327 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 15:06:21.659611 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522a77fe-9180-4987-91c8-92934c00cc63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:06:21.660268 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522a77fe-9180-4987-91c8-92934c00cc63 admin admin] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:06:21.660653 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522a77fe-9180-4987-91c8-92934c00cc63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:06:21.660947 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522a77fe-9180-4987-91c8-92934c00cc63 admin admin] PATH_INFO: `/v3/users` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:06:21.661717 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522a77fe-9180-4987-91c8-92934c00cc63 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:06:21.663680 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522a77fe-9180-4987-91c8-92934c00cc63 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:06:21.692012 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1392/2777] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Sep 1 15:06:21 2022] POST /identity/v3/users => generated 299 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 15:06:21.693467 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5612fd53-50d2-4414-a7ab-ea27cb35de89 admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:06:21.700855 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 15:06:21.705574 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5612fd53-50d2-4414-a7ab-ea27cb35de89 admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:06:21.712398 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e65e3bf-3660-4d6d-964e-f683d6e636b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:06:21.713350 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e65e3bf-3660-4d6d-964e-f683d6e636b7 admin admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:06:21.714019 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e65e3bf-3660-4d6d-964e-f683d6e636b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:06:21.714671 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e65e3bf-3660-4d6d-964e-f683d6e636b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:06:21.715395 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e65e3bf-3660-4d6d-964e-f683d6e636b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:06:21.717850 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e65e3bf-3660-4d6d-964e-f683d6e636b7 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:06:21.731820 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1386/2778] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 1 15:06:21 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 15:06:21.735550 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522a77fe-9180-4987-91c8-92934c00cc63 admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:06:21.757869 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522a77fe-9180-4987-91c8-92934c00cc63 admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:06:21.764938 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458aa877-53ad-4cb4-a988-43cff6516d3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:06:21.765729 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458aa877-53ad-4cb4-a988-43cff6516d3e admin admin] REQUEST_METHOD: `PUT` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:06:21.766016 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458aa877-53ad-4cb4-a988-43cff6516d3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:06:21.766403 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458aa877-53ad-4cb4-a988-43cff6516d3e admin admin] PATH_INFO: `/v3/projects/c6250d0754c84f6ca21aae74258da08b/users/6240f3e7020e46da863871949cab8a06/roles/2a2adb769e55476b82e5014897e0a9c0` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:06:21.786137 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458aa877-53ad-4cb4-a988-43cff6516d3e admin admin] RBAC: Authorizing `identity:create_grant(project_id=c6250d0754c84f6ca21aae74258da08b, user_id=6240f3e7020e46da863871949cab8a06, role_id=2a2adb769e55476b82e5014897e0a9c0)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:06:21.789638 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458aa877-53ad-4cb4-a988-43cff6516d3e admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:06:21.806768 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1393/2779] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 1 15:06:21 2022] PUT /identity/v3/projects/c6250d0754c84f6ca21aae74258da08b/users/6240f3e7020e46da863871949cab8a06/roles/2a2adb769e55476b82e5014897e0a9c0 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 15:06:21.811846 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e65e3bf-3660-4d6d-964e-f683d6e636b7 admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:06:21.821973 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e65e3bf-3660-4d6d-964e-f683d6e636b7 admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:06:21.891585 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458aa877-53ad-4cb4-a988-43cff6516d3e admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:06:21.893546 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-458aa877-53ad-4cb4-a988-43cff6516d3e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:06:21.923671 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19bec32-7d56-4aec-82d9-b2e89bab7d7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:06:21.924913 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19bec32-7d56-4aec-82d9-b2e89bab7d7e admin admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:06:21.925212 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19bec32-7d56-4aec-82d9-b2e89bab7d7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:06:21.925560 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19bec32-7d56-4aec-82d9-b2e89bab7d7e admin admin] PATH_INFO: `/v3/roles` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:06:21.926494 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19bec32-7d56-4aec-82d9-b2e89bab7d7e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:06:21.928350 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19bec32-7d56-4aec-82d9-b2e89bab7d7e admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:06:21.957035 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1387/2780] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 1 15:06:21 2022] GET /identity/v3/roles => generated 1420 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 15:06:21.958205 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19bec32-7d56-4aec-82d9-b2e89bab7d7e admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:06:21.969536 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19bec32-7d56-4aec-82d9-b2e89bab7d7e admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:06:21.976743 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4d2ca0-b690-49dc-a15e-9226ba265a59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:06:21.977415 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4d2ca0-b690-49dc-a15e-9226ba265a59 admin admin] REQUEST_METHOD: `PUT` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:06:21.977699 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4d2ca0-b690-49dc-a15e-9226ba265a59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:06:21.977978 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4d2ca0-b690-49dc-a15e-9226ba265a59 admin admin] PATH_INFO: `/v3/projects/c6250d0754c84f6ca21aae74258da08b/users/6240f3e7020e46da863871949cab8a06/roles/2a2adb769e55476b82e5014897e0a9c0` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:06:21.980035 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4d2ca0-b690-49dc-a15e-9226ba265a59 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c6250d0754c84f6ca21aae74258da08b, user_id=6240f3e7020e46da863871949cab8a06, role_id=2a2adb769e55476b82e5014897e0a9c0)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:06:21.993769 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4d2ca0-b690-49dc-a15e-9226ba265a59 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:06:22.015469 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458aa877-53ad-4cb4-a988-43cff6516d3e admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:06:22.021043 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1388/2781] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 1 15:06:21 2022] PUT /identity/v3/projects/c6250d0754c84f6ca21aae74258da08b/users/6240f3e7020e46da863871949cab8a06/roles/2a2adb769e55476b82e5014897e0a9c0 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 15:06:22.127778 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556b830f-3734-4a5c-b2db-ba182fc48268 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:06:22.128481 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556b830f-3734-4a5c-b2db-ba182fc48268 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:06:22.129050 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556b830f-3734-4a5c-b2db-ba182fc48268 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:06:22.129372 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556b830f-3734-4a5c-b2db-ba182fc48268 admin admin] PATH_INFO: `/v3/projects/0669894d406746cbb39ad80d2c5548bd` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:06:22.131008 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556b830f-3734-4a5c-b2db-ba182fc48268 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0669894d406746cbb39ad80d2c5548bd)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:06:22.133773 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556b830f-3734-4a5c-b2db-ba182fc48268 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:06:22.199014 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-556b830f-3734-4a5c-b2db-ba182fc48268 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0669894d406746cbb39ad80d2c5548bd', 'request_id': 'req-556b830f-3734-4a5c-b2db-ba182fc48268'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:06:22.206588 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-556b830f-3734-4a5c-b2db-ba182fc48268 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0669894d406746cbb39ad80d2c5548bd', 'request_id': 'req-556b830f-3734-4a5c-b2db-ba182fc48268'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:06:22.219359 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-556b830f-3734-4a5c-b2db-ba182fc48268 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0669894d406746cbb39ad80d2c5548bd', 'request_id': 'req-556b830f-3734-4a5c-b2db-ba182fc48268'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:06:22.230832 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 15:06:22.236449 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-556b830f-3734-4a5c-b2db-ba182fc48268 admin admin] The token cache is being invalidate because project 0669894d406746cbb39ad80d2c5548bd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81342) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 15:06:22.237129 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-556b830f-3734-4a5c-b2db-ba182fc48268 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:06:22.238167 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-556b830f-3734-4a5c-b2db-ba182fc48268 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:06:22.240789 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1394/2782] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Sep 1 15:06:21 2022] DELETE /identity/v3/projects/0669894d406746cbb39ad80d2c5548bd => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 15:06:24.995001 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddfd4a9-5a88-4c83-b4dd-5ed80df7c086 None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:06:24.995444 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddfd4a9-5a88-4c83-b4dd-5ed80df7c086 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:06:24.995567 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddfd4a9-5a88-4c83-b4dd-5ed80df7c086 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:06:25.027289 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-5ddfd4a9-5a88-4c83-b4dd-5ed80df7c086 None None] MFA Rules not processed for user `27653498cd0a4e089f1cbc43470182ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 15:06:25.028627 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-5ddfd4a9-5a88-4c83-b4dd-5ed80df7c086 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:06:25.122199 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1389/2783] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 15:06:24 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 15:06:28.207838 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae43b12-ec74-44d8-a353-d0ead6e497ef None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:06:28.208372 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae43b12-ec74-44d8-a353-d0ead6e497ef None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:06:28.208372 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae43b12-ec74-44d8-a353-d0ead6e497ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:06:28.258991 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-3ae43b12-ec74-44d8-a353-d0ead6e497ef None None] MFA Rules not processed for user `6240f3e7020e46da863871949cab8a06`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 15:06:28.260302 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-3ae43b12-ec74-44d8-a353-d0ead6e497ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:06:28.398060 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1395/2784] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 15:06:28 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 15:06:28.413015 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddfd4a9-5a88-4c83-b4dd-5ed80df7c086 None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:06:28.414765 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-5ddfd4a9-5a88-4c83-b4dd-5ed80df7c086 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:06:28.599954 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddfd4a9-5a88-4c83-b4dd-5ed80df7c086 None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:06:28.607155 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35180a2-ed48-475a-a537-0d64079ad0a1 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3069f72ce98049b99c9d98229a62bb51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:06:28.607961 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35180a2-ed48-475a-a537-0d64079ad0a1 service glance] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:06:28.608415 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35180a2-ed48-475a-a537-0d64079ad0a1 service glance] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:06:28.608851 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35180a2-ed48-475a-a537-0d64079ad0a1 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:06:28.611223 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-b35180a2-ed48-475a-a537-0d64079ad0a1 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:06:28.620514 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 15:06:28.738919 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35180a2-ed48-475a-a537-0d64079ad0a1 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:06:28.740643 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35180a2-ed48-475a-a537-0d64079ad0a1 service glance] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:06:28.752269 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1390/2785] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 15:06:28 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 15:06:29.014698 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c469060-0b7d-4a8b-93b9-df4211829b48 None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:06:29.015353 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c469060-0b7d-4a8b-93b9-df4211829b48 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:06:29.015838 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c469060-0b7d-4a8b-93b9-df4211829b48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:06:29.091873 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-0c469060-0b7d-4a8b-93b9-df4211829b48 None None] MFA Rules not processed for user `24b6f275ffb04ffeae3c2097bfd040f5`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 15:06:29.093097 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-0c469060-0b7d-4a8b-93b9-df4211829b48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:06:29.196088 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1396/2786] 10.222.0.33 () {58 vars in 1009 bytes} [Thu Sep 1 15:06:29 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 15:06:29.257183 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35180a2-ed48-475a-a537-0d64079ad0a1 service glance] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:06:29.258856 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-b35180a2-ed48-475a-a537-0d64079ad0a1 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:06:29.341418 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35180a2-ed48-475a-a537-0d64079ad0a1 service glance] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:06:29.347908 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90ed957-6b53-4b05-9f1d-dcf405b50494 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73a5bda5a6ea46e9863ccafd57dee339', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:06:29.348560 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90ed957-6b53-4b05-9f1d-dcf405b50494 service swift] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:06:29.348916 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90ed957-6b53-4b05-9f1d-dcf405b50494 service swift] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:06:29.349394 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90ed957-6b53-4b05-9f1d-dcf405b50494 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:06:29.358657 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90ed957-6b53-4b05-9f1d-dcf405b50494 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:06:29.360282 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90ed957-6b53-4b05-9f1d-dcf405b50494 service swift] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:06:29.369273 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1391/2787] 10.222.0.33 () {60 vars in 1384 bytes} [Thu Sep 1 15:06:29 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 01 15:06:38.838170 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9eceaf-4068-4c23-b95d-e6f3cfa48a6b None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:06:38.838831 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9eceaf-4068-4c23-b95d-e6f3cfa48a6b None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:06:38.838970 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9eceaf-4068-4c23-b95d-e6f3cfa48a6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:06:38.859128 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-8e9eceaf-4068-4c23-b95d-e6f3cfa48a6b None None] MFA Rules not processed for user `70cd6c18012049419d7fa7b591b4bd28`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 15:06:38.860469 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-8e9eceaf-4068-4c23-b95d-e6f3cfa48a6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:06:38.867737 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1397/2788] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 15:06:38 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 15:06:39.365750 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=56 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 15:06:40.454849 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90ed957-6b53-4b05-9f1d-dcf405b50494 service swift] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:06:40.456874 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-a90ed957-6b53-4b05-9f1d-dcf405b50494 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:06:40.522724 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9eceaf-4068-4c23-b95d-e6f3cfa48a6b None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:06:40.525157 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-8e9eceaf-4068-4c23-b95d-e6f3cfa48a6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:06:40.573583 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90ed957-6b53-4b05-9f1d-dcf405b50494 service swift] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:06:40.579716 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750f8e34-d9c8-4b17-b4b0-7889277a935f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f312c8dcabe48ef9ea453364ade38ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:06:40.580690 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750f8e34-d9c8-4b17-b4b0-7889277a935f service placement] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:06:40.581179 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750f8e34-d9c8-4b17-b4b0-7889277a935f service placement] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:06:40.581674 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750f8e34-d9c8-4b17-b4b0-7889277a935f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:06:40.583692 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-750f8e34-d9c8-4b17-b4b0-7889277a935f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:06:40.598022 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750f8e34-d9c8-4b17-b4b0-7889277a935f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:06:40.601438 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750f8e34-d9c8-4b17-b4b0-7889277a935f service placement] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:06:40.605447 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 15:06:40.622281 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1392/2789] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 15:06:40 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 15:06:40.666398 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9eceaf-4068-4c23-b95d-e6f3cfa48a6b None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:06:40.673369 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fc957d-4adf-48ab-8198-155f849e19bc service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f312c8dcabe48ef9ea453364ade38ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:06:40.674118 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fc957d-4adf-48ab-8198-155f849e19bc service placement] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:06:40.674520 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fc957d-4adf-48ab-8198-155f849e19bc service placement] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:06:40.674867 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fc957d-4adf-48ab-8198-155f849e19bc service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:06:40.686024 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fc957d-4adf-48ab-8198-155f849e19bc service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:06:40.687772 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fc957d-4adf-48ab-8198-155f849e19bc service placement] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:06:40.700051 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1398/2790] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 15:06:40 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 15:06:43.078820 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afe4546-7af0-4e45-bfaa-29446ac9ca8d None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:06:43.079407 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afe4546-7af0-4e45-bfaa-29446ac9ca8d None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:06:43.079407 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afe4546-7af0-4e45-bfaa-29446ac9ca8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:06:43.116015 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-4afe4546-7af0-4e45-bfaa-29446ac9ca8d None None] MFA Rules not processed for user `24b6f275ffb04ffeae3c2097bfd040f5`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 15:06:43.117590 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-4afe4546-7af0-4e45-bfaa-29446ac9ca8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:06:43.128233 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1393/2791] 10.222.0.33 () {58 vars in 1009 bytes} [Thu Sep 1 15:06:43 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 15:06:43.155165 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fc957d-4adf-48ab-8198-155f849e19bc service placement] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:06:43.169121 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fc957d-4adf-48ab-8198-155f849e19bc service placement] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:06:43.175950 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64d2e80-e813-4196-88d0-8cc7cd4f5c83 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73a5bda5a6ea46e9863ccafd57dee339', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:06:43.176382 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64d2e80-e813-4196-88d0-8cc7cd4f5c83 service swift] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:06:43.176658 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64d2e80-e813-4196-88d0-8cc7cd4f5c83 service swift] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:06:43.176925 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64d2e80-e813-4196-88d0-8cc7cd4f5c83 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:06:43.187167 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64d2e80-e813-4196-88d0-8cc7cd4f5c83 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:06:43.188712 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64d2e80-e813-4196-88d0-8cc7cd4f5c83 service swift] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:06:43.198576 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1399/2792] 10.222.0.33 () {60 vars in 1384 bytes} [Thu Sep 1 15:06:43 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 01 15:06:53.110204 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=48 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 15:06:53.193842 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=18 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 15:07:10.410739 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afe4546-7af0-4e45-bfaa-29446ac9ca8d None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:07:10.412159 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-4afe4546-7af0-4e45-bfaa-29446ac9ca8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:07:10.487651 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afe4546-7af0-4e45-bfaa-29446ac9ca8d None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:07:10.491224 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a282be8-3ebe-4653-85e1-2b5eb5ddcdf3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2790149639aa418bbc4e4982e93eda44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:07:10.491821 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a282be8-3ebe-4653-85e1-2b5eb5ddcdf3 service neutron] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:07:10.492093 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a282be8-3ebe-4653-85e1-2b5eb5ddcdf3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:07:10.492355 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a282be8-3ebe-4653-85e1-2b5eb5ddcdf3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:07:10.501586 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a282be8-3ebe-4653-85e1-2b5eb5ddcdf3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:07:10.503035 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a282be8-3ebe-4653-85e1-2b5eb5ddcdf3 service neutron] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:07:10.512133 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1394/2793] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 15:07:10 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 15:07:20.508135 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=35 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 15:07:40.824129 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64d2e80-e813-4196-88d0-8cc7cd4f5c83 service swift] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:07:40.826635 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-a64d2e80-e813-4196-88d0-8cc7cd4f5c83 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:07:40.975929 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64d2e80-e813-4196-88d0-8cc7cd4f5c83 service swift] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:07:40.983527 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f07b93-65a9-4618-8966-4dee88ef38d1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3b81739e34e4e648f88b6d73fcb5e46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:07:40.985160 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f07b93-65a9-4618-8966-4dee88ef38d1 service cinder] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:07:40.985583 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f07b93-65a9-4618-8966-4dee88ef38d1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:07:40.985953 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f07b93-65a9-4618-8966-4dee88ef38d1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:07:40.990776 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-98f07b93-65a9-4618-8966-4dee88ef38d1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:07:41.095880 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f07b93-65a9-4618-8966-4dee88ef38d1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:07:41.097328 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f07b93-65a9-4618-8966-4dee88ef38d1 service cinder] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:07:41.108331 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1400/2794] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 15:07:40 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 284 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 15:07:43.349148 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd036d2e-7658-4497-9eb0-851dbd48dc92 None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:07:43.349989 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd036d2e-7658-4497-9eb0-851dbd48dc92 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:07:43.350446 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd036d2e-7658-4497-9eb0-851dbd48dc92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:07:43.371440 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-dd036d2e-7658-4497-9eb0-851dbd48dc92 None None] MFA Rules not processed for user `afa6212f31034e27a8d32f114937190e`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 15:07:43.373628 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-dd036d2e-7658-4497-9eb0-851dbd48dc92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:07:43.383575 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1395/2795] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 15:07:43 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 15:07:44.411087 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c2a4dd-a745-4d37-906a-33c0dbc22a39 None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:07:44.411484 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c2a4dd-a745-4d37-906a-33c0dbc22a39 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:07:44.411719 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c2a4dd-a745-4d37-906a-33c0dbc22a39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:07:44.433618 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-b4c2a4dd-a745-4d37-906a-33c0dbc22a39 None None] MFA Rules not processed for user `27653498cd0a4e089f1cbc43470182ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 15:07:44.435017 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-b4c2a4dd-a745-4d37-906a-33c0dbc22a39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:07:44.443608 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1401/2796] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 15:07:44 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 15:07:53.365745 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=3 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 15:07:54.427884 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=60 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 15:08:11.373334 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd036d2e-7658-4497-9eb0-851dbd48dc92 None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:08:11.375632 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-dd036d2e-7658-4497-9eb0-851dbd48dc92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:08:11.475377 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd036d2e-7658-4497-9eb0-851dbd48dc92 None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:08:11.480027 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0055d83-ee0d-4994-9613-8e6ac295c87c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:08:11.480657 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0055d83-ee0d-4994-9613-8e6ac295c87c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:08:11.480939 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0055d83-ee0d-4994-9613-8e6ac295c87c admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:08:11.481214 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0055d83-ee0d-4994-9613-8e6ac295c87c admin admin] PATH_INFO: `/v3/users/210bc3fc36724b32ba50364332c6f900` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:08:11.482444 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0055d83-ee0d-4994-9613-8e6ac295c87c admin admin] RBAC: Authorizing `identity:delete_user(user_id=210bc3fc36724b32ba50364332c6f900)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:08:11.484385 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0055d83-ee0d-4994-9613-8e6ac295c87c admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:08:11.543904 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-e0055d83-ee0d-4994-9613-8e6ac295c87c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '210bc3fc36724b32ba50364332c6f900'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:08:11.551314 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-e0055d83-ee0d-4994-9613-8e6ac295c87c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '210bc3fc36724b32ba50364332c6f900'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:08:11.551931 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-e0055d83-ee0d-4994-9613-8e6ac295c87c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '210bc3fc36724b32ba50364332c6f900'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:08:11.567392 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-e0055d83-ee0d-4994-9613-8e6ac295c87c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '210bc3fc36724b32ba50364332c6f900'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:08:11.567392 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-e0055d83-ee0d-4994-9613-8e6ac295c87c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '210bc3fc36724b32ba50364332c6f900'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:08:11.578900 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1396/2797] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Sep 1 15:08:11 2022] DELETE /identity/v3/users/210bc3fc36724b32ba50364332c6f900 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 15:08:11.757440 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c2a4dd-a745-4d37-906a-33c0dbc22a39 None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:08:11.773011 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c2a4dd-a745-4d37-906a-33c0dbc22a39 None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:08:11.834452 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230751f1-efa5-417e-b172-ed053ee99c99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:08:11.834998 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230751f1-efa5-417e-b172-ed053ee99c99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:08:11.835275 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230751f1-efa5-417e-b172-ed053ee99c99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:08:11.835543 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230751f1-efa5-417e-b172-ed053ee99c99 admin admin] PATH_INFO: `/v3/projects/764329f7a06b42969616baffd95cbd29` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:08:11.836708 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230751f1-efa5-417e-b172-ed053ee99c99 admin admin] RBAC: Authorizing `identity:delete_project(project_id=764329f7a06b42969616baffd95cbd29)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:08:11.838686 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230751f1-efa5-417e-b172-ed053ee99c99 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:08:11.884807 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-230751f1-efa5-417e-b172-ed053ee99c99 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '764329f7a06b42969616baffd95cbd29', 'request_id': 'req-230751f1-efa5-417e-b172-ed053ee99c99'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:08:11.891207 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-230751f1-efa5-417e-b172-ed053ee99c99 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '764329f7a06b42969616baffd95cbd29', 'request_id': 'req-230751f1-efa5-417e-b172-ed053ee99c99'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:08:11.898844 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-230751f1-efa5-417e-b172-ed053ee99c99 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '764329f7a06b42969616baffd95cbd29', 'request_id': 'req-230751f1-efa5-417e-b172-ed053ee99c99'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:08:11.909454 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-230751f1-efa5-417e-b172-ed053ee99c99 admin admin] The token cache is being invalidate because project 764329f7a06b42969616baffd95cbd29 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81342) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 15:08:11.909745 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-230751f1-efa5-417e-b172-ed053ee99c99 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:08:11.910418 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-230751f1-efa5-417e-b172-ed053ee99c99 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:08:11.911948 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1402/2798] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Sep 1 15:08:11 2022] DELETE /identity/v3/projects/764329f7a06b42969616baffd95cbd29 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 15:08:11.916098 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0055d83-ee0d-4994-9613-8e6ac295c87c admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:08:11.917509 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-e0055d83-ee0d-4994-9613-8e6ac295c87c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:08:11.931291 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0055d83-ee0d-4994-9613-8e6ac295c87c admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:08:11.977585 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 15:08:11.989047 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baac5764-8cb3-4ccc-a906-c8f5f55d48e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:08:11.989685 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baac5764-8cb3-4ccc-a906-c8f5f55d48e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:08:11.989961 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baac5764-8cb3-4ccc-a906-c8f5f55d48e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:08:11.990252 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baac5764-8cb3-4ccc-a906-c8f5f55d48e9 admin admin] PATH_INFO: `/v3/users/ebda997302e24adb962d50bad531ed63` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:08:11.991435 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baac5764-8cb3-4ccc-a906-c8f5f55d48e9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ebda997302e24adb962d50bad531ed63)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:08:11.993369 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baac5764-8cb3-4ccc-a906-c8f5f55d48e9 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:08:12.040427 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-baac5764-8cb3-4ccc-a906-c8f5f55d48e9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ebda997302e24adb962d50bad531ed63'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:08:12.048660 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-baac5764-8cb3-4ccc-a906-c8f5f55d48e9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ebda997302e24adb962d50bad531ed63'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:08:12.049313 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-baac5764-8cb3-4ccc-a906-c8f5f55d48e9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ebda997302e24adb962d50bad531ed63'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:08:12.061817 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-baac5764-8cb3-4ccc-a906-c8f5f55d48e9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ebda997302e24adb962d50bad531ed63'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:08:12.062531 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-baac5764-8cb3-4ccc-a906-c8f5f55d48e9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ebda997302e24adb962d50bad531ed63'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:08:12.075107 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1397/2799] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Sep 1 15:08:11 2022] DELETE /identity/v3/users/ebda997302e24adb962d50bad531ed63 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 15:08:12.250279 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230751f1-efa5-417e-b172-ed053ee99c99 admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:08:12.251756 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-230751f1-efa5-417e-b172-ed053ee99c99 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:08:12.265011 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230751f1-efa5-417e-b172-ed053ee99c99 admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:08:12.324205 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f21e84a-ac6a-4f7d-896e-6d04769c21a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:08:12.324848 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f21e84a-ac6a-4f7d-896e-6d04769c21a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:08:12.325155 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f21e84a-ac6a-4f7d-896e-6d04769c21a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:08:12.325429 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f21e84a-ac6a-4f7d-896e-6d04769c21a6 admin admin] PATH_INFO: `/v3/projects/0063eabaf1ec4cd5bd120ca3db0622af` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:08:12.326623 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f21e84a-ac6a-4f7d-896e-6d04769c21a6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0063eabaf1ec4cd5bd120ca3db0622af)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:08:12.328551 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f21e84a-ac6a-4f7d-896e-6d04769c21a6 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:08:12.364473 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 15:08:12.378114 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-5f21e84a-ac6a-4f7d-896e-6d04769c21a6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0063eabaf1ec4cd5bd120ca3db0622af', 'request_id': 'req-5f21e84a-ac6a-4f7d-896e-6d04769c21a6'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:08:12.384228 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-5f21e84a-ac6a-4f7d-896e-6d04769c21a6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0063eabaf1ec4cd5bd120ca3db0622af', 'request_id': 'req-5f21e84a-ac6a-4f7d-896e-6d04769c21a6'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:08:12.391821 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-5f21e84a-ac6a-4f7d-896e-6d04769c21a6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0063eabaf1ec4cd5bd120ca3db0622af', 'request_id': 'req-5f21e84a-ac6a-4f7d-896e-6d04769c21a6'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:08:12.402760 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-5f21e84a-ac6a-4f7d-896e-6d04769c21a6 admin admin] The token cache is being invalidate because project 0063eabaf1ec4cd5bd120ca3db0622af was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81342) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 15:08:12.403050 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-5f21e84a-ac6a-4f7d-896e-6d04769c21a6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:08:12.404180 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-5f21e84a-ac6a-4f7d-896e-6d04769c21a6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:08:12.405672 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1403/2800] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Sep 1 15:08:12 2022] DELETE /identity/v3/projects/0063eabaf1ec4cd5bd120ca3db0622af => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 15:08:22.077153 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=30,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 15:08:22.399728 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=12,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 15:08:31.332107 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baac5764-8cb3-4ccc-a906-c8f5f55d48e9 admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:08:31.333651 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-baac5764-8cb3-4ccc-a906-c8f5f55d48e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:08:31.390987 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baac5764-8cb3-4ccc-a906-c8f5f55d48e9 admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:08:31.439252 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71032d42-d209-4f22-af61-abe0c0031efe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2790149639aa418bbc4e4982e93eda44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:08:31.439870 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71032d42-d209-4f22-af61-abe0c0031efe service neutron] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:08:31.440145 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71032d42-d209-4f22-af61-abe0c0031efe service neutron] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:08:31.440413 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71032d42-d209-4f22-af61-abe0c0031efe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:08:31.442192 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-71032d42-d209-4f22-af61-abe0c0031efe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:08:31.523051 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71032d42-d209-4f22-af61-abe0c0031efe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:08:31.524509 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71032d42-d209-4f22-af61-abe0c0031efe service neutron] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:08:31.545363 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1398/2801] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 15:08:31 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 15:08:36.255487 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f21e84a-ac6a-4f7d-896e-6d04769c21a6 admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:08:36.256892 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-5f21e84a-ac6a-4f7d-896e-6d04769c21a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:08:36.316143 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f21e84a-ac6a-4f7d-896e-6d04769c21a6 admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:08:36.319982 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8985872a-ac05-4fc1-a852-21c254fc2233 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3b81739e34e4e648f88b6d73fcb5e46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:08:36.320785 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8985872a-ac05-4fc1-a852-21c254fc2233 service cinder] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:08:36.321070 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8985872a-ac05-4fc1-a852-21c254fc2233 service cinder] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:08:36.321342 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8985872a-ac05-4fc1-a852-21c254fc2233 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:08:36.323111 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-8985872a-ac05-4fc1-a852-21c254fc2233 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:08:36.337270 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8985872a-ac05-4fc1-a852-21c254fc2233 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:08:36.338781 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8985872a-ac05-4fc1-a852-21c254fc2233 service cinder] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:08:36.350437 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1404/2802] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 15:08:36 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 15:08:39.728104 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0971c5c4-cb3a-48fd-8c04-1680ea2935b2 None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:08:39.728617 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0971c5c4-cb3a-48fd-8c04-1680ea2935b2 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:08:39.728617 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0971c5c4-cb3a-48fd-8c04-1680ea2935b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:08:39.779176 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-0971c5c4-cb3a-48fd-8c04-1680ea2935b2 None None] MFA Rules not processed for user `24b6f275ffb04ffeae3c2097bfd040f5`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 15:08:39.780378 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-0971c5c4-cb3a-48fd-8c04-1680ea2935b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:08:39.788734 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 15:08:39.844708 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1399/2803] 10.222.0.33 () {58 vars in 967 bytes} [Thu Sep 1 15:08:39 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 15:08:39.892299 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8985872a-ac05-4fc1-a852-21c254fc2233 service cinder] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:08:39.893870 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-8985872a-ac05-4fc1-a852-21c254fc2233 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:08:39.990275 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8985872a-ac05-4fc1-a852-21c254fc2233 service cinder] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:08:39.995390 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6372850c-0efd-40af-b41b-70db18ad7b0e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73a5bda5a6ea46e9863ccafd57dee339', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:08:39.995999 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6372850c-0efd-40af-b41b-70db18ad7b0e service swift] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:08:39.996279 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6372850c-0efd-40af-b41b-70db18ad7b0e service swift] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:08:39.996550 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6372850c-0efd-40af-b41b-70db18ad7b0e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:08:40.004990 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6372850c-0efd-40af-b41b-70db18ad7b0e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:08:40.006662 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6372850c-0efd-40af-b41b-70db18ad7b0e service swift] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:08:40.014988 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1405/2804] 10.222.0.33 () {60 vars in 1384 bytes} [Thu Sep 1 15:08:39 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 01 15:08:46.732371 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0971c5c4-cb3a-48fd-8c04-1680ea2935b2 None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:08:46.734276 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-0971c5c4-cb3a-48fd-8c04-1680ea2935b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:08:46.822532 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0971c5c4-cb3a-48fd-8c04-1680ea2935b2 None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:08:46.827098 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3dd404-6499-4122-b691-b28dd77f2c20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:08:46.827711 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3dd404-6499-4122-b691-b28dd77f2c20 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:08:46.827996 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3dd404-6499-4122-b691-b28dd77f2c20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:08:46.828268 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3dd404-6499-4122-b691-b28dd77f2c20 admin admin] PATH_INFO: `/v3/users/70cd6c18012049419d7fa7b591b4bd28` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:08:46.829399 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3dd404-6499-4122-b691-b28dd77f2c20 admin admin] RBAC: Authorizing `identity:delete_user(user_id=70cd6c18012049419d7fa7b591b4bd28)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:08:46.831383 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3dd404-6499-4122-b691-b28dd77f2c20 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:08:46.880926 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-5c3dd404-6499-4122-b691-b28dd77f2c20 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '70cd6c18012049419d7fa7b591b4bd28'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:08:46.888429 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-5c3dd404-6499-4122-b691-b28dd77f2c20 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '70cd6c18012049419d7fa7b591b4bd28'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:08:46.889091 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-5c3dd404-6499-4122-b691-b28dd77f2c20 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '70cd6c18012049419d7fa7b591b4bd28'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:08:46.901450 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-5c3dd404-6499-4122-b691-b28dd77f2c20 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '70cd6c18012049419d7fa7b591b4bd28'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:08:46.902117 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-5c3dd404-6499-4122-b691-b28dd77f2c20 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '70cd6c18012049419d7fa7b591b4bd28'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:08:46.914622 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1400/2805] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Sep 1 15:08:46 2022] DELETE /identity/v3/users/70cd6c18012049419d7fa7b591b4bd28 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 15:08:47.094889 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6372850c-0efd-40af-b41b-70db18ad7b0e service swift] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:08:47.106477 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6372850c-0efd-40af-b41b-70db18ad7b0e service swift] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:08:47.169260 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04763eb-150a-40b9-a1a3-c7b8c4762b37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:08:47.169890 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04763eb-150a-40b9-a1a3-c7b8c4762b37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:08:47.170171 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04763eb-150a-40b9-a1a3-c7b8c4762b37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:08:47.170486 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04763eb-150a-40b9-a1a3-c7b8c4762b37 admin admin] PATH_INFO: `/v3/projects/a7311840457040a992b3bebaa2d32843` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:08:47.171637 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04763eb-150a-40b9-a1a3-c7b8c4762b37 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a7311840457040a992b3bebaa2d32843)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:08:47.173551 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04763eb-150a-40b9-a1a3-c7b8c4762b37 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:08:47.191453 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 15:08:47.225090 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-d04763eb-150a-40b9-a1a3-c7b8c4762b37 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a7311840457040a992b3bebaa2d32843', 'request_id': 'req-d04763eb-150a-40b9-a1a3-c7b8c4762b37'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:08:47.231215 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-d04763eb-150a-40b9-a1a3-c7b8c4762b37 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a7311840457040a992b3bebaa2d32843', 'request_id': 'req-d04763eb-150a-40b9-a1a3-c7b8c4762b37'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:08:47.241982 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-d04763eb-150a-40b9-a1a3-c7b8c4762b37 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a7311840457040a992b3bebaa2d32843', 'request_id': 'req-d04763eb-150a-40b9-a1a3-c7b8c4762b37'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:08:47.252469 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-d04763eb-150a-40b9-a1a3-c7b8c4762b37 admin admin] The token cache is being invalidate because project a7311840457040a992b3bebaa2d32843 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81342) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 15:08:47.252751 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-d04763eb-150a-40b9-a1a3-c7b8c4762b37 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:08:47.253383 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-d04763eb-150a-40b9-a1a3-c7b8c4762b37 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:08:47.254894 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1406/2806] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Sep 1 15:08:47 2022] DELETE /identity/v3/projects/a7311840457040a992b3bebaa2d32843 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 15:08:47.258982 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3dd404-6499-4122-b691-b28dd77f2c20 admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:08:47.260366 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-5c3dd404-6499-4122-b691-b28dd77f2c20 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:08:47.273683 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3dd404-6499-4122-b691-b28dd77f2c20 admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:08:47.300044 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 15:08:47.333359 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e05e9c4-28c9-4324-b5ef-ab99021c6c87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:08:47.333995 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e05e9c4-28c9-4324-b5ef-ab99021c6c87 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:08:47.334296 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e05e9c4-28c9-4324-b5ef-ab99021c6c87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:08:47.334567 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e05e9c4-28c9-4324-b5ef-ab99021c6c87 admin admin] PATH_INFO: `/v3/users/6240f3e7020e46da863871949cab8a06` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:08:47.335737 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e05e9c4-28c9-4324-b5ef-ab99021c6c87 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6240f3e7020e46da863871949cab8a06)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:08:47.337653 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e05e9c4-28c9-4324-b5ef-ab99021c6c87 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:08:47.385662 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-8e05e9c4-28c9-4324-b5ef-ab99021c6c87 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6240f3e7020e46da863871949cab8a06'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:08:47.394223 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-8e05e9c4-28c9-4324-b5ef-ab99021c6c87 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6240f3e7020e46da863871949cab8a06'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:08:47.394890 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-8e05e9c4-28c9-4324-b5ef-ab99021c6c87 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6240f3e7020e46da863871949cab8a06'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:08:47.408493 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-8e05e9c4-28c9-4324-b5ef-ab99021c6c87 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6240f3e7020e46da863871949cab8a06'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:08:47.409218 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-8e05e9c4-28c9-4324-b5ef-ab99021c6c87 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6240f3e7020e46da863871949cab8a06'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:08:47.422308 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1401/2807] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Sep 1 15:08:47 2022] DELETE /identity/v3/users/6240f3e7020e46da863871949cab8a06 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 15:08:47.640340 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04763eb-150a-40b9-a1a3-c7b8c4762b37 admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:08:47.641921 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-d04763eb-150a-40b9-a1a3-c7b8c4762b37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:08:47.655122 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04763eb-150a-40b9-a1a3-c7b8c4762b37 admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:08:47.721146 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bb787e-de0e-45e3-a40d-51d4cdc88b3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:08:47.721818 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bb787e-de0e-45e3-a40d-51d4cdc88b3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:08:47.722098 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bb787e-de0e-45e3-a40d-51d4cdc88b3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:08:47.722394 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bb787e-de0e-45e3-a40d-51d4cdc88b3d admin admin] PATH_INFO: `/v3/projects/c6250d0754c84f6ca21aae74258da08b` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:08:47.723552 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bb787e-de0e-45e3-a40d-51d4cdc88b3d admin admin] RBAC: Authorizing `identity:delete_project(project_id=c6250d0754c84f6ca21aae74258da08b)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:08:47.725457 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bb787e-de0e-45e3-a40d-51d4cdc88b3d admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:08:47.777717 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-89bb787e-de0e-45e3-a40d-51d4cdc88b3d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c6250d0754c84f6ca21aae74258da08b', 'request_id': 'req-89bb787e-de0e-45e3-a40d-51d4cdc88b3d'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:08:47.784816 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-89bb787e-de0e-45e3-a40d-51d4cdc88b3d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c6250d0754c84f6ca21aae74258da08b', 'request_id': 'req-89bb787e-de0e-45e3-a40d-51d4cdc88b3d'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:08:47.793126 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-89bb787e-de0e-45e3-a40d-51d4cdc88b3d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c6250d0754c84f6ca21aae74258da08b', 'request_id': 'req-89bb787e-de0e-45e3-a40d-51d4cdc88b3d'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:08:47.805392 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-89bb787e-de0e-45e3-a40d-51d4cdc88b3d admin admin] The token cache is being invalidate because project c6250d0754c84f6ca21aae74258da08b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81342) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 15:08:47.805861 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-89bb787e-de0e-45e3-a40d-51d4cdc88b3d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:08:47.806753 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-89bb787e-de0e-45e3-a40d-51d4cdc88b3d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:08:47.808382 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1407/2808] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Sep 1 15:08:47 2022] DELETE /identity/v3/projects/c6250d0754c84f6ca21aae74258da08b => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 15:08:57.150108 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e05e9c4-28c9-4324-b5ef-ab99021c6c87 admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:08:57.154347 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-8e05e9c4-28c9-4324-b5ef-ab99021c6c87 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:08:57.173226 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e05e9c4-28c9-4324-b5ef-ab99021c6c87 admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:08:57.232625 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523688ff-ff6b-40a5-a64b-c7abb6c576ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:08:57.233249 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523688ff-ff6b-40a5-a64b-c7abb6c576ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:08:57.233525 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523688ff-ff6b-40a5-a64b-c7abb6c576ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:08:57.233794 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523688ff-ff6b-40a5-a64b-c7abb6c576ac admin admin] PATH_INFO: `/v3/users/27653498cd0a4e089f1cbc43470182ee` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:08:57.234921 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523688ff-ff6b-40a5-a64b-c7abb6c576ac admin admin] RBAC: Authorizing `identity:delete_user(user_id=27653498cd0a4e089f1cbc43470182ee)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:08:57.236819 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523688ff-ff6b-40a5-a64b-c7abb6c576ac admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:08:57.284788 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-523688ff-ff6b-40a5-a64b-c7abb6c576ac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '27653498cd0a4e089f1cbc43470182ee'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:08:57.287318 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 15:08:57.294092 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-523688ff-ff6b-40a5-a64b-c7abb6c576ac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '27653498cd0a4e089f1cbc43470182ee'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:08:57.294999 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-523688ff-ff6b-40a5-a64b-c7abb6c576ac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '27653498cd0a4e089f1cbc43470182ee'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:08:57.307988 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-523688ff-ff6b-40a5-a64b-c7abb6c576ac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '27653498cd0a4e089f1cbc43470182ee'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:08:57.308557 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-523688ff-ff6b-40a5-a64b-c7abb6c576ac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '27653498cd0a4e089f1cbc43470182ee'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:08:57.320671 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1402/2809] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Sep 1 15:08:57 2022] DELETE /identity/v3/users/27653498cd0a4e089f1cbc43470182ee => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 15:08:57.489562 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bb787e-de0e-45e3-a40d-51d4cdc88b3d admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:08:57.491040 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-89bb787e-de0e-45e3-a40d-51d4cdc88b3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:08:57.509422 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bb787e-de0e-45e3-a40d-51d4cdc88b3d admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:08:57.540756 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 15:08:57.570463 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8c13a6-4bfd-4a04-9d23-fdb74bdc90bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:08:57.571100 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8c13a6-4bfd-4a04-9d23-fdb74bdc90bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:08:57.571380 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8c13a6-4bfd-4a04-9d23-fdb74bdc90bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:08:57.571655 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8c13a6-4bfd-4a04-9d23-fdb74bdc90bd admin admin] PATH_INFO: `/v3/projects/0b789cb75bc24a898888246a7745537e` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:08:57.572798 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8c13a6-4bfd-4a04-9d23-fdb74bdc90bd admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b789cb75bc24a898888246a7745537e)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:08:57.575259 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8c13a6-4bfd-4a04-9d23-fdb74bdc90bd admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:08:57.621852 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-be8c13a6-4bfd-4a04-9d23-fdb74bdc90bd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b789cb75bc24a898888246a7745537e', 'request_id': 'req-be8c13a6-4bfd-4a04-9d23-fdb74bdc90bd'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:08:57.627989 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-be8c13a6-4bfd-4a04-9d23-fdb74bdc90bd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b789cb75bc24a898888246a7745537e', 'request_id': 'req-be8c13a6-4bfd-4a04-9d23-fdb74bdc90bd'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:08:57.635640 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-be8c13a6-4bfd-4a04-9d23-fdb74bdc90bd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b789cb75bc24a898888246a7745537e', 'request_id': 'req-be8c13a6-4bfd-4a04-9d23-fdb74bdc90bd'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:08:57.646555 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-be8c13a6-4bfd-4a04-9d23-fdb74bdc90bd admin admin] The token cache is being invalidate because project 0b789cb75bc24a898888246a7745537e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81342) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 15:08:57.646844 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-be8c13a6-4bfd-4a04-9d23-fdb74bdc90bd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:08:57.647475 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-be8c13a6-4bfd-4a04-9d23-fdb74bdc90bd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:08:57.648965 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1408/2810] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Sep 1 15:08:57 2022] DELETE /identity/v3/projects/0b789cb75bc24a898888246a7745537e => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 15:08:57.654570 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba21761-922d-41bf-b2f0-5fd0990e5b4c None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:08:57.654945 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba21761-922d-41bf-b2f0-5fd0990e5b4c None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:08:57.655300 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba21761-922d-41bf-b2f0-5fd0990e5b4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:08:57.713229 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-0ba21761-922d-41bf-b2f0-5fd0990e5b4c None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 15:08:57.714475 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-0ba21761-922d-41bf-b2f0-5fd0990e5b4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:08:57.772775 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1403/2811] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 15:08:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 15:08:57.781315 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae8b2c9-708f-40de-8ad1-ee018266f6f6 None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:08:57.781604 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae8b2c9-708f-40de-8ad1-ee018266f6f6 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:08:57.781878 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae8b2c9-708f-40de-8ad1-ee018266f6f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:08:57.801941 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-2ae8b2c9-708f-40de-8ad1-ee018266f6f6 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 15:08:57.803185 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-2ae8b2c9-708f-40de-8ad1-ee018266f6f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:08:57.811287 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1409/2812] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 15:08:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 15:08:57.815695 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba21761-922d-41bf-b2f0-5fd0990e5b4c None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:08:57.825988 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba21761-922d-41bf-b2f0-5fd0990e5b4c None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:08:57.830491 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d846498-c0c7-40fa-b1e5-803bfce8f1fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:08:57.831081 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d846498-c0c7-40fa-b1e5-803bfce8f1fe admin admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:08:57.831357 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d846498-c0c7-40fa-b1e5-803bfce8f1fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:08:57.831628 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d846498-c0c7-40fa-b1e5-803bfce8f1fe admin admin] PATH_INFO: `/v3/domains` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:08:57.832296 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d846498-c0c7-40fa-b1e5-803bfce8f1fe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81341) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 15:08:57.832594 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d846498-c0c7-40fa-b1e5-803bfce8f1fe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:08:57.834197 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d846498-c0c7-40fa-b1e5-803bfce8f1fe admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:08:57.846445 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1404/2813] 10.222.0.33 () {58 vars in 1153 bytes} [Thu Sep 1 15:08:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 15:08:57.850760 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae8b2c9-708f-40de-8ad1-ee018266f6f6 None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:08:57.860147 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae8b2c9-708f-40de-8ad1-ee018266f6f6 None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:08:57.865018 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011ccfb3-4a66-4146-a747-34297d96e46b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:08:57.865640 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011ccfb3-4a66-4146-a747-34297d96e46b admin admin] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:08:57.865913 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011ccfb3-4a66-4146-a747-34297d96e46b admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:08:57.866182 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011ccfb3-4a66-4146-a747-34297d96e46b admin admin] PATH_INFO: `/v3/projects` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:08:57.866841 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011ccfb3-4a66-4146-a747-34297d96e46b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:08:57.868727 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011ccfb3-4a66-4146-a747-34297d96e46b admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:08:57.905722 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1410/2814] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Sep 1 15:08:57 2022] POST /identity/v3/projects => generated 357 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 15:08:57.910019 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d846498-c0c7-40fa-b1e5-803bfce8f1fe admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:08:57.920438 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d846498-c0c7-40fa-b1e5-803bfce8f1fe admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:08:57.976026 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937e6801-a7df-49ef-b4f0-f3d76e04e11f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:08:57.976623 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937e6801-a7df-49ef-b4f0-f3d76e04e11f admin admin] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:08:57.976898 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937e6801-a7df-49ef-b4f0-f3d76e04e11f admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:08:57.977166 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937e6801-a7df-49ef-b4f0-f3d76e04e11f admin admin] PATH_INFO: `/v3/users` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:08:57.977803 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937e6801-a7df-49ef-b4f0-f3d76e04e11f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:08:57.979741 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937e6801-a7df-49ef-b4f0-f3d76e04e11f admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:08:58.001169 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1405/2815] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Sep 1 15:08:57 2022] POST /identity/v3/users => generated 279 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 15:08:58.005975 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011ccfb3-4a66-4146-a747-34297d96e46b admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:08:58.017134 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011ccfb3-4a66-4146-a747-34297d96e46b admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:08:58.021670 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcd79dd-3f24-4c61-9581-870b29ee57af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:08:58.022277 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcd79dd-3f24-4c61-9581-870b29ee57af admin admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:08:58.022577 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcd79dd-3f24-4c61-9581-870b29ee57af admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:08:58.022824 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcd79dd-3f24-4c61-9581-870b29ee57af admin admin] PATH_INFO: `/v3/roles` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:08:58.023461 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdcd79dd-3f24-4c61-9581-870b29ee57af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:08:58.025045 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdcd79dd-3f24-4c61-9581-870b29ee57af admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:08:58.035063 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1411/2816] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 1 15:08:58 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 15:08:58.039327 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937e6801-a7df-49ef-b4f0-f3d76e04e11f admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:08:58.049668 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937e6801-a7df-49ef-b4f0-f3d76e04e11f admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:08:58.054271 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f684df8a-b85b-4dba-afbb-5731198d7794 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:08:58.054873 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f684df8a-b85b-4dba-afbb-5731198d7794 admin admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:08:58.055151 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f684df8a-b85b-4dba-afbb-5731198d7794 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:08:58.055418 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f684df8a-b85b-4dba-afbb-5731198d7794 admin admin] PATH_INFO: `/v3/roles` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:08:58.056053 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f684df8a-b85b-4dba-afbb-5731198d7794 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:08:58.057685 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f684df8a-b85b-4dba-afbb-5731198d7794 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:08:58.068750 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1406/2817] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 1 15:08:58 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 15:08:58.073050 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcd79dd-3f24-4c61-9581-870b29ee57af admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:08:58.083696 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcd79dd-3f24-4c61-9581-870b29ee57af admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:08:58.088293 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c39c40-f33f-4daf-801f-b594c5fd8f25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:08:58.088937 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c39c40-f33f-4daf-801f-b594c5fd8f25 admin admin] REQUEST_METHOD: `PUT` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:08:58.089214 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c39c40-f33f-4daf-801f-b594c5fd8f25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:08:58.089483 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c39c40-f33f-4daf-801f-b594c5fd8f25 admin admin] PATH_INFO: `/v3/projects/f852dea2cca841f5a37b8a19f442a1cb/users/ccd19286e4454fdab7421d4b5c6589db/roles/5d1feef0e13d4874928bd86258c4e3b2` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:08:58.104772 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c39c40-f33f-4daf-801f-b594c5fd8f25 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f852dea2cca841f5a37b8a19f442a1cb, user_id=ccd19286e4454fdab7421d4b5c6589db, role_id=5d1feef0e13d4874928bd86258c4e3b2)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:08:58.107654 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c39c40-f33f-4daf-801f-b594c5fd8f25 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:08:58.120673 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1412/2818] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 1 15:08:58 2022] PUT /identity/v3/projects/f852dea2cca841f5a37b8a19f442a1cb/users/ccd19286e4454fdab7421d4b5c6589db/roles/5d1feef0e13d4874928bd86258c4e3b2 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 15:08:58.132375 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc15bbd-5af6-4b43-80ff-1286a6003150 None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:08:58.132655 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc15bbd-5af6-4b43-80ff-1286a6003150 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:08:58.132925 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc15bbd-5af6-4b43-80ff-1286a6003150 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:08:58.140868 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:DELETE=3,keystone:SELECT=94,keystone:INSERT=3 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 15:08:58.165334 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-5cc15bbd-5af6-4b43-80ff-1286a6003150 None None] MFA Rules not processed for user `ccd19286e4454fdab7421d4b5c6589db`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 15:08:58.166630 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-5cc15bbd-5af6-4b43-80ff-1286a6003150 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:08:58.235615 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1407/2819] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 15:08:58 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 15:08:58.240217 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c39c40-f33f-4daf-801f-b594c5fd8f25 admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:08:58.291322 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 15:08:58.304219 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c39c40-f33f-4daf-801f-b594c5fd8f25 admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:08:58.309508 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26dd2ff1-2902-467e-ba94-8c93b1aaf002 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:08:58.310108 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dd2ff1-2902-467e-ba94-8c93b1aaf002 admin admin] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:08:58.310413 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dd2ff1-2902-467e-ba94-8c93b1aaf002 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:08:58.310707 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dd2ff1-2902-467e-ba94-8c93b1aaf002 admin admin] PATH_INFO: `/v3/projects` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:08:58.311334 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26dd2ff1-2902-467e-ba94-8c93b1aaf002 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:08:58.313234 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26dd2ff1-2902-467e-ba94-8c93b1aaf002 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:08:58.339180 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1413/2820] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Sep 1 15:08:58 2022] POST /identity/v3/projects => generated 357 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 15:08:58.343202 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc15bbd-5af6-4b43-80ff-1286a6003150 None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:08:58.353769 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc15bbd-5af6-4b43-80ff-1286a6003150 None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:08:58.408745 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6985b643-c79b-43d9-9d98-ed925994c54a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:08:58.409337 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6985b643-c79b-43d9-9d98-ed925994c54a admin admin] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:08:58.409616 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6985b643-c79b-43d9-9d98-ed925994c54a admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:08:58.409878 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6985b643-c79b-43d9-9d98-ed925994c54a admin admin] PATH_INFO: `/v3/users` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:08:58.410535 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6985b643-c79b-43d9-9d98-ed925994c54a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:08:58.412455 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6985b643-c79b-43d9-9d98-ed925994c54a admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:08:58.432889 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1408/2821] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Sep 1 15:08:58 2022] POST /identity/v3/users => generated 285 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 15:08:58.437434 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26dd2ff1-2902-467e-ba94-8c93b1aaf002 admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:08:58.447643 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26dd2ff1-2902-467e-ba94-8c93b1aaf002 admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:08:58.452233 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9e8635-8f98-4578-8b34-74386e883502 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:08:58.452825 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9e8635-8f98-4578-8b34-74386e883502 admin admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:08:58.453105 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9e8635-8f98-4578-8b34-74386e883502 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:08:58.453381 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9e8635-8f98-4578-8b34-74386e883502 admin admin] PATH_INFO: `/v3/roles` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:08:58.454024 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9e8635-8f98-4578-8b34-74386e883502 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:08:58.455677 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9e8635-8f98-4578-8b34-74386e883502 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:08:58.466315 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1414/2822] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 1 15:08:58 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 15:08:58.469993 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6985b643-c79b-43d9-9d98-ed925994c54a admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:08:58.479701 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6985b643-c79b-43d9-9d98-ed925994c54a admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:08:58.484757 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75ae066-3e8f-4a13-8cb3-eed146dfc11e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:08:58.485379 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75ae066-3e8f-4a13-8cb3-eed146dfc11e admin admin] REQUEST_METHOD: `PUT` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:08:58.485654 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75ae066-3e8f-4a13-8cb3-eed146dfc11e admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:08:58.485920 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75ae066-3e8f-4a13-8cb3-eed146dfc11e admin admin] PATH_INFO: `/v3/projects/d11c7eb9312849cf90413cc4d7761a61/users/65d39d350e2443379b1fe1856ff7b96f/roles/2a2adb769e55476b82e5014897e0a9c0` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:08:58.501001 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75ae066-3e8f-4a13-8cb3-eed146dfc11e admin admin] RBAC: Authorizing `identity:create_grant(project_id=d11c7eb9312849cf90413cc4d7761a61, user_id=65d39d350e2443379b1fe1856ff7b96f, role_id=2a2adb769e55476b82e5014897e0a9c0)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:08:58.503725 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75ae066-3e8f-4a13-8cb3-eed146dfc11e admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:08:58.516289 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1409/2823] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 1 15:08:58 2022] PUT /identity/v3/projects/d11c7eb9312849cf90413cc4d7761a61/users/65d39d350e2443379b1fe1856ff7b96f/roles/2a2adb769e55476b82e5014897e0a9c0 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 15:08:58.520111 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9e8635-8f98-4578-8b34-74386e883502 admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:08:58.529869 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9e8635-8f98-4578-8b34-74386e883502 admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:08:58.586218 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27977f60-ea70-41c2-b633-81e0abf3b62d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:08:58.586852 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27977f60-ea70-41c2-b633-81e0abf3b62d admin admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:08:58.587164 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27977f60-ea70-41c2-b633-81e0abf3b62d admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:08:58.587439 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27977f60-ea70-41c2-b633-81e0abf3b62d admin admin] PATH_INFO: `/v3/roles` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:08:58.588106 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27977f60-ea70-41c2-b633-81e0abf3b62d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:08:58.589755 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27977f60-ea70-41c2-b633-81e0abf3b62d admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:08:58.599360 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1415/2824] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 1 15:08:58 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 15:08:58.603511 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75ae066-3e8f-4a13-8cb3-eed146dfc11e admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:08:58.614380 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75ae066-3e8f-4a13-8cb3-eed146dfc11e admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:08:58.618919 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364cea43-7195-4c91-a0c4-f0b62e5434d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:08:58.619603 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364cea43-7195-4c91-a0c4-f0b62e5434d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:08:58.619887 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364cea43-7195-4c91-a0c4-f0b62e5434d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:08:58.620157 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364cea43-7195-4c91-a0c4-f0b62e5434d4 admin admin] PATH_INFO: `/v3/projects/d11c7eb9312849cf90413cc4d7761a61/users/65d39d350e2443379b1fe1856ff7b96f/roles/2a2adb769e55476b82e5014897e0a9c0` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:08:58.622707 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364cea43-7195-4c91-a0c4-f0b62e5434d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d11c7eb9312849cf90413cc4d7761a61, user_id=65d39d350e2443379b1fe1856ff7b96f, role_id=2a2adb769e55476b82e5014897e0a9c0)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:08:58.625343 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364cea43-7195-4c91-a0c4-f0b62e5434d4 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:08:58.635686 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1410/2825] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 1 15:08:58 2022] PUT /identity/v3/projects/d11c7eb9312849cf90413cc4d7761a61/users/65d39d350e2443379b1fe1856ff7b96f/roles/2a2adb769e55476b82e5014897e0a9c0 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 15:08:58.645886 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f117d14d-cdd5-49b3-8a83-040604f37e82 None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:08:58.646172 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f117d14d-cdd5-49b3-8a83-040604f37e82 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:08:58.646478 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f117d14d-cdd5-49b3-8a83-040604f37e82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:08:58.675128 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-f117d14d-cdd5-49b3-8a83-040604f37e82 None None] MFA Rules not processed for user `65d39d350e2443379b1fe1856ff7b96f`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 15:08:58.676329 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-f117d14d-cdd5-49b3-8a83-040604f37e82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:08:58.744107 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1416/2826] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 15:08:58 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 15:08:58.755232 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364cea43-7195-4c91-a0c4-f0b62e5434d4 admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:08:58.756951 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-364cea43-7195-4c91-a0c4-f0b62e5434d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:08:58.805297 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 15:08:58.816876 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364cea43-7195-4c91-a0c4-f0b62e5434d4 admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:08:58.820456 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d17162-d9ec-4b23-803b-68a08981c9fc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3b81739e34e4e648f88b6d73fcb5e46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:08:58.821039 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d17162-d9ec-4b23-803b-68a08981c9fc service cinder] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:08:58.821314 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d17162-d9ec-4b23-803b-68a08981c9fc service cinder] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:08:58.821580 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d17162-d9ec-4b23-803b-68a08981c9fc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:08:58.830444 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d17162-d9ec-4b23-803b-68a08981c9fc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:08:58.831922 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d17162-d9ec-4b23-803b-68a08981c9fc service cinder] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:08:58.841858 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1411/2827] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 15:08:58 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 15:08:58.915131 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f117d14d-cdd5-49b3-8a83-040604f37e82 None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:08:58.925732 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f117d14d-cdd5-49b3-8a83-040604f37e82 None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:08:58.929307 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f36ea76-a877-4095-afe3-b097726fd37b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3b81739e34e4e648f88b6d73fcb5e46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:08:58.929902 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f36ea76-a877-4095-afe3-b097726fd37b service cinder] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:08:58.930318 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f36ea76-a877-4095-afe3-b097726fd37b service cinder] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:08:58.930591 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f36ea76-a877-4095-afe3-b097726fd37b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:08:58.974581 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 15:08:58.987976 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f36ea76-a877-4095-afe3-b097726fd37b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:08:58.989419 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f36ea76-a877-4095-afe3-b097726fd37b service cinder] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:08:58.998177 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1417/2828] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 15:08:58 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 15:09:08.837121 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=6 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 15:09:08.994252 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=7 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 15:09:13.779897 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d17162-d9ec-4b23-803b-68a08981c9fc service cinder] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:09:13.842546 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d17162-d9ec-4b23-803b-68a08981c9fc service cinder] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:09:13.847098 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e04bc9-c7a0-411e-a9fd-b4a273ba53d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:09:13.847727 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e04bc9-c7a0-411e-a9fd-b4a273ba53d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:09:13.848003 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e04bc9-c7a0-411e-a9fd-b4a273ba53d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:09:13.848271 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e04bc9-c7a0-411e-a9fd-b4a273ba53d5 admin admin] PATH_INFO: `/v3/users/ccd19286e4454fdab7421d4b5c6589db` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:09:13.849393 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e04bc9-c7a0-411e-a9fd-b4a273ba53d5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ccd19286e4454fdab7421d4b5c6589db)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:09:13.851375 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e04bc9-c7a0-411e-a9fd-b4a273ba53d5 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:09:13.897473 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-56e04bc9-c7a0-411e-a9fd-b4a273ba53d5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ccd19286e4454fdab7421d4b5c6589db'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:09:13.904800 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-56e04bc9-c7a0-411e-a9fd-b4a273ba53d5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ccd19286e4454fdab7421d4b5c6589db'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:09:13.905506 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-56e04bc9-c7a0-411e-a9fd-b4a273ba53d5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ccd19286e4454fdab7421d4b5c6589db'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:09:13.917568 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-56e04bc9-c7a0-411e-a9fd-b4a273ba53d5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ccd19286e4454fdab7421d4b5c6589db'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:09:13.918287 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-56e04bc9-c7a0-411e-a9fd-b4a273ba53d5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ccd19286e4454fdab7421d4b5c6589db'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:09:13.931210 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1412/2829] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Sep 1 15:09:13 2022] DELETE /identity/v3/users/ccd19286e4454fdab7421d4b5c6589db => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 15:09:13.935727 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82e313b-acfb-4010-b1ba-1c752e2ffefc None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:09:13.936695 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82e313b-acfb-4010-b1ba-1c752e2ffefc None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:09:13.936810 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82e313b-acfb-4010-b1ba-1c752e2ffefc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:09:13.956362 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-a82e313b-acfb-4010-b1ba-1c752e2ffefc None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 15:09:13.957658 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-a82e313b-acfb-4010-b1ba-1c752e2ffefc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:09:14.016114 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1418/2830] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 15:09:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 15:09:14.025821 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e04bc9-c7a0-411e-a9fd-b4a273ba53d5 admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:09:14.027251 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-56e04bc9-c7a0-411e-a9fd-b4a273ba53d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:09:14.083684 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e04bc9-c7a0-411e-a9fd-b4a273ba53d5 admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:09:14.087374 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f2608a-c192-471a-81c4-e8a728d43256 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2790149639aa418bbc4e4982e93eda44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:09:14.087969 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f2608a-c192-471a-81c4-e8a728d43256 service neutron] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:09:14.088248 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f2608a-c192-471a-81c4-e8a728d43256 service neutron] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:09:14.088520 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f2608a-c192-471a-81c4-e8a728d43256 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:09:14.097535 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f2608a-c192-471a-81c4-e8a728d43256 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:09:14.099013 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f2608a-c192-471a-81c4-e8a728d43256 service neutron] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:09:14.109066 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1413/2831] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 15:09:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 15:09:14.377626 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82e313b-acfb-4010-b1ba-1c752e2ffefc None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:09:14.379296 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-a82e313b-acfb-4010-b1ba-1c752e2ffefc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:09:14.392406 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82e313b-acfb-4010-b1ba-1c752e2ffefc None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:09:14.397183 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdf8bae-b03f-4a2e-8b5d-47865685f20c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:09:14.397910 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdf8bae-b03f-4a2e-8b5d-47865685f20c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:09:14.398196 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdf8bae-b03f-4a2e-8b5d-47865685f20c admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:09:14.398524 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdf8bae-b03f-4a2e-8b5d-47865685f20c admin admin] PATH_INFO: `/v3/projects/f852dea2cca841f5a37b8a19f442a1cb` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:09:14.399790 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdf8bae-b03f-4a2e-8b5d-47865685f20c admin admin] RBAC: Authorizing `identity:delete_project(project_id=f852dea2cca841f5a37b8a19f442a1cb)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:09:14.401590 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdf8bae-b03f-4a2e-8b5d-47865685f20c admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:09:14.450024 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-efdf8bae-b03f-4a2e-8b5d-47865685f20c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f852dea2cca841f5a37b8a19f442a1cb', 'request_id': 'req-efdf8bae-b03f-4a2e-8b5d-47865685f20c'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:09:14.456133 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-efdf8bae-b03f-4a2e-8b5d-47865685f20c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f852dea2cca841f5a37b8a19f442a1cb', 'request_id': 'req-efdf8bae-b03f-4a2e-8b5d-47865685f20c'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:09:14.463574 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-efdf8bae-b03f-4a2e-8b5d-47865685f20c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f852dea2cca841f5a37b8a19f442a1cb', 'request_id': 'req-efdf8bae-b03f-4a2e-8b5d-47865685f20c'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:09:14.474381 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-efdf8bae-b03f-4a2e-8b5d-47865685f20c admin admin] The token cache is being invalidate because project f852dea2cca841f5a37b8a19f442a1cb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81342) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 15:09:14.474671 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-efdf8bae-b03f-4a2e-8b5d-47865685f20c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:09:14.475312 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-efdf8bae-b03f-4a2e-8b5d-47865685f20c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:09:14.476778 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1419/2832] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Sep 1 15:09:14 2022] DELETE /identity/v3/projects/f852dea2cca841f5a37b8a19f442a1cb => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 15:09:14.480804 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f2608a-c192-471a-81c4-e8a728d43256 service neutron] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:09:14.490877 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f2608a-c192-471a-81c4-e8a728d43256 service neutron] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:09:14.508193 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 15:09:14.549840 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd26062b-568e-4822-b76d-b5a6e6496273 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:09:14.550433 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd26062b-568e-4822-b76d-b5a6e6496273 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:09:14.550715 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd26062b-568e-4822-b76d-b5a6e6496273 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:09:14.550986 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd26062b-568e-4822-b76d-b5a6e6496273 admin admin] PATH_INFO: `/v3/users/65d39d350e2443379b1fe1856ff7b96f` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:09:14.552091 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd26062b-568e-4822-b76d-b5a6e6496273 admin admin] RBAC: Authorizing `identity:delete_user(user_id=65d39d350e2443379b1fe1856ff7b96f)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:09:14.554009 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd26062b-568e-4822-b76d-b5a6e6496273 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:09:14.597395 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-bd26062b-568e-4822-b76d-b5a6e6496273 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '65d39d350e2443379b1fe1856ff7b96f'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:09:14.604623 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-bd26062b-568e-4822-b76d-b5a6e6496273 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '65d39d350e2443379b1fe1856ff7b96f'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:09:14.605346 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-bd26062b-568e-4822-b76d-b5a6e6496273 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '65d39d350e2443379b1fe1856ff7b96f'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:09:14.617523 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-bd26062b-568e-4822-b76d-b5a6e6496273 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '65d39d350e2443379b1fe1856ff7b96f'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:09:14.618170 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-bd26062b-568e-4822-b76d-b5a6e6496273 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '65d39d350e2443379b1fe1856ff7b96f'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:09:14.630157 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1414/2833] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Sep 1 15:09:14 2022] DELETE /identity/v3/users/65d39d350e2443379b1fe1856ff7b96f => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 15:09:14.902542 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdf8bae-b03f-4a2e-8b5d-47865685f20c admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:09:14.903979 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-efdf8bae-b03f-4a2e-8b5d-47865685f20c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:09:14.918053 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdf8bae-b03f-4a2e-8b5d-47865685f20c admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:09:14.975019 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2fc69e-3bc6-495c-8395-9bd6f9c36a58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:09:14.975641 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2fc69e-3bc6-495c-8395-9bd6f9c36a58 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:09:14.975909 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2fc69e-3bc6-495c-8395-9bd6f9c36a58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:09:14.976178 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2fc69e-3bc6-495c-8395-9bd6f9c36a58 admin admin] PATH_INFO: `/v3/projects/d11c7eb9312849cf90413cc4d7761a61` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:09:14.977317 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2fc69e-3bc6-495c-8395-9bd6f9c36a58 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d11c7eb9312849cf90413cc4d7761a61)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:09:14.979324 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2fc69e-3bc6-495c-8395-9bd6f9c36a58 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:09:15.008384 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 15:09:15.025069 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-8e2fc69e-3bc6-495c-8395-9bd6f9c36a58 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd11c7eb9312849cf90413cc4d7761a61', 'request_id': 'req-8e2fc69e-3bc6-495c-8395-9bd6f9c36a58'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:09:15.031879 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-8e2fc69e-3bc6-495c-8395-9bd6f9c36a58 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd11c7eb9312849cf90413cc4d7761a61', 'request_id': 'req-8e2fc69e-3bc6-495c-8395-9bd6f9c36a58'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:09:15.038987 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-8e2fc69e-3bc6-495c-8395-9bd6f9c36a58 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd11c7eb9312849cf90413cc4d7761a61', 'request_id': 'req-8e2fc69e-3bc6-495c-8395-9bd6f9c36a58'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:09:15.049355 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-8e2fc69e-3bc6-495c-8395-9bd6f9c36a58 admin admin] The token cache is being invalidate because project d11c7eb9312849cf90413cc4d7761a61 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81342) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 15:09:15.049641 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-8e2fc69e-3bc6-495c-8395-9bd6f9c36a58 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:09:15.050291 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-8e2fc69e-3bc6-495c-8395-9bd6f9c36a58 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:09:15.051748 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1420/2834] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Sep 1 15:09:14 2022] DELETE /identity/v3/projects/d11c7eb9312849cf90413cc4d7761a61 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 15:09:15.056694 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1139c83-89c6-4a44-a752-2cacd955c369 None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:09:15.057153 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1139c83-89c6-4a44-a752-2cacd955c369 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:09:15.057252 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1139c83-89c6-4a44-a752-2cacd955c369 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:09:15.078039 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-c1139c83-89c6-4a44-a752-2cacd955c369 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 15:09:15.079341 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-c1139c83-89c6-4a44-a752-2cacd955c369 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:09:15.136675 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1415/2835] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 15:09:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 15:09:15.146577 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f47d910-f814-4aeb-b0d5-6556c1b02ea6 None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:09:15.146865 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f47d910-f814-4aeb-b0d5-6556c1b02ea6 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:09:15.147161 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f47d910-f814-4aeb-b0d5-6556c1b02ea6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:09:15.167097 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-1f47d910-f814-4aeb-b0d5-6556c1b02ea6 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 15:09:15.168302 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-1f47d910-f814-4aeb-b0d5-6556c1b02ea6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:09:15.176285 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1421/2836] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 15:09:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 15:09:15.180430 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1139c83-89c6-4a44-a752-2cacd955c369 None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:09:15.190639 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1139c83-89c6-4a44-a752-2cacd955c369 None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:09:15.195155 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2519e94-c3f1-4db4-9e2b-5ed415c0a9ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:09:15.195739 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2519e94-c3f1-4db4-9e2b-5ed415c0a9ab admin admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:09:15.196013 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2519e94-c3f1-4db4-9e2b-5ed415c0a9ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:09:15.196280 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2519e94-c3f1-4db4-9e2b-5ed415c0a9ab admin admin] PATH_INFO: `/v3/domains` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:09:15.196921 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2519e94-c3f1-4db4-9e2b-5ed415c0a9ab admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81341) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 15:09:15.197217 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2519e94-c3f1-4db4-9e2b-5ed415c0a9ab admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:09:15.198857 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2519e94-c3f1-4db4-9e2b-5ed415c0a9ab admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:09:15.210762 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1416/2837] 10.222.0.33 () {58 vars in 1153 bytes} [Thu Sep 1 15:09:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 15:09:15.214971 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f47d910-f814-4aeb-b0d5-6556c1b02ea6 None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:09:15.223987 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f47d910-f814-4aeb-b0d5-6556c1b02ea6 None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:09:15.228551 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7403df-479c-45a7-baf1-8b2c764a63b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:09:15.229134 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7403df-479c-45a7-baf1-8b2c764a63b5 admin admin] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:09:15.229409 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7403df-479c-45a7-baf1-8b2c764a63b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:09:15.229674 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7403df-479c-45a7-baf1-8b2c764a63b5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:09:15.230333 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7403df-479c-45a7-baf1-8b2c764a63b5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:09:15.232221 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7403df-479c-45a7-baf1-8b2c764a63b5 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:09:15.258909 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1422/2838] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Sep 1 15:09:15 2022] POST /identity/v3/projects => generated 385 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 15:09:15.263040 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2519e94-c3f1-4db4-9e2b-5ed415c0a9ab admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:09:15.273356 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2519e94-c3f1-4db4-9e2b-5ed415c0a9ab admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:09:15.298414 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 15:09:15.331576 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cb8f12-13c7-4b5e-a839-e6f0dc50c942 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:09:15.332110 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cb8f12-13c7-4b5e-a839-e6f0dc50c942 admin admin] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:09:15.332392 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cb8f12-13c7-4b5e-a839-e6f0dc50c942 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:09:15.332665 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cb8f12-13c7-4b5e-a839-e6f0dc50c942 admin admin] PATH_INFO: `/v3/users` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:09:15.333287 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36cb8f12-13c7-4b5e-a839-e6f0dc50c942 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:09:15.335260 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36cb8f12-13c7-4b5e-a839-e6f0dc50c942 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:09:15.355840 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1417/2839] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Sep 1 15:09:15 2022] POST /identity/v3/users => generated 293 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 15:09:15.360012 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7403df-479c-45a7-baf1-8b2c764a63b5 admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:09:15.376698 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7403df-479c-45a7-baf1-8b2c764a63b5 admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:09:15.381793 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736782ef-d5a0-4d41-b660-f1f4c4d7c1b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:09:15.382424 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736782ef-d5a0-4d41-b660-f1f4c4d7c1b0 admin admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:09:15.382705 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736782ef-d5a0-4d41-b660-f1f4c4d7c1b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:09:15.382976 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736782ef-d5a0-4d41-b660-f1f4c4d7c1b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:09:15.383622 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736782ef-d5a0-4d41-b660-f1f4c4d7c1b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:09:15.385258 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736782ef-d5a0-4d41-b660-f1f4c4d7c1b0 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:09:15.395895 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1423/2840] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 1 15:09:15 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 15:09:15.399909 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cb8f12-13c7-4b5e-a839-e6f0dc50c942 admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:09:15.409837 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cb8f12-13c7-4b5e-a839-e6f0dc50c942 admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:09:15.414464 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4c4100-7229-4032-ac89-2868bc9d2151 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:09:15.415058 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4c4100-7229-4032-ac89-2868bc9d2151 admin admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:09:15.415336 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4c4100-7229-4032-ac89-2868bc9d2151 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:09:15.415612 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4c4100-7229-4032-ac89-2868bc9d2151 admin admin] PATH_INFO: `/v3/roles` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:09:15.416282 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4c4100-7229-4032-ac89-2868bc9d2151 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:09:15.418162 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4c4100-7229-4032-ac89-2868bc9d2151 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:09:15.428772 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1418/2841] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 1 15:09:15 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 15:09:15.433120 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736782ef-d5a0-4d41-b660-f1f4c4d7c1b0 admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:09:15.444945 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736782ef-d5a0-4d41-b660-f1f4c4d7c1b0 admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:09:15.450300 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08839923-12f0-4daf-87a0-88ea3179e54b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:09:15.450960 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08839923-12f0-4daf-87a0-88ea3179e54b admin admin] REQUEST_METHOD: `PUT` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:09:15.451241 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08839923-12f0-4daf-87a0-88ea3179e54b admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:09:15.451516 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08839923-12f0-4daf-87a0-88ea3179e54b admin admin] PATH_INFO: `/v3/projects/46e1fdef6476415f9bd507856d625f28/users/5a704949e57a4e48a05174c607340cc7/roles/5d1feef0e13d4874928bd86258c4e3b2` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:09:15.467442 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08839923-12f0-4daf-87a0-88ea3179e54b admin admin] RBAC: Authorizing `identity:create_grant(project_id=46e1fdef6476415f9bd507856d625f28, user_id=5a704949e57a4e48a05174c607340cc7, role_id=5d1feef0e13d4874928bd86258c4e3b2)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:09:15.470097 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08839923-12f0-4daf-87a0-88ea3179e54b admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:09:15.483526 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1424/2842] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 1 15:09:15 2022] PUT /identity/v3/projects/46e1fdef6476415f9bd507856d625f28/users/5a704949e57a4e48a05174c607340cc7/roles/5d1feef0e13d4874928bd86258c4e3b2 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 15:09:15.495366 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7d59bd-152d-44c9-baa7-a0f5d227b72b None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:09:15.495647 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7d59bd-152d-44c9-baa7-a0f5d227b72b None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:09:15.496091 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7d59bd-152d-44c9-baa7-a0f5d227b72b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:09:15.524383 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-7c7d59bd-152d-44c9-baa7-a0f5d227b72b None None] MFA Rules not processed for user `5a704949e57a4e48a05174c607340cc7`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 15:09:15.525624 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-7c7d59bd-152d-44c9-baa7-a0f5d227b72b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:09:15.591839 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1419/2843] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 15:09:15 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 15:09:15.596529 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08839923-12f0-4daf-87a0-88ea3179e54b admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:09:15.660005 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08839923-12f0-4daf-87a0-88ea3179e54b admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:09:15.664919 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a54f262-db4b-480a-84fc-7402937dcc19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:09:15.665533 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a54f262-db4b-480a-84fc-7402937dcc19 admin admin] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:09:15.665812 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a54f262-db4b-480a-84fc-7402937dcc19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:09:15.666088 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a54f262-db4b-480a-84fc-7402937dcc19 admin admin] PATH_INFO: `/v3/projects` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:09:15.667030 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a54f262-db4b-480a-84fc-7402937dcc19 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:09:15.668949 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a54f262-db4b-480a-84fc-7402937dcc19 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:09:15.697688 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1425/2844] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Sep 1 15:09:15 2022] POST /identity/v3/projects => generated 387 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 15:09:15.701960 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7d59bd-152d-44c9-baa7-a0f5d227b72b None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:09:15.712637 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7d59bd-152d-44c9-baa7-a0f5d227b72b None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:09:15.768651 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cb20c9-d7a0-4774-9ce5-f59f304e696e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:09:15.769259 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cb20c9-d7a0-4774-9ce5-f59f304e696e admin admin] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:09:15.769538 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cb20c9-d7a0-4774-9ce5-f59f304e696e admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:09:15.769808 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cb20c9-d7a0-4774-9ce5-f59f304e696e admin admin] PATH_INFO: `/v3/users` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:09:15.770476 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68cb20c9-d7a0-4774-9ce5-f59f304e696e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:09:15.772387 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68cb20c9-d7a0-4774-9ce5-f59f304e696e admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:09:15.793757 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1420/2845] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Sep 1 15:09:15 2022] POST /identity/v3/users => generated 300 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 15:09:15.798344 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a54f262-db4b-480a-84fc-7402937dcc19 admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:09:15.808542 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a54f262-db4b-480a-84fc-7402937dcc19 admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:09:15.813674 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56897625-9e54-46b1-866a-e865bcb039ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:09:15.814365 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56897625-9e54-46b1-866a-e865bcb039ac admin admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:09:15.814651 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56897625-9e54-46b1-866a-e865bcb039ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:09:15.814917 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56897625-9e54-46b1-866a-e865bcb039ac admin admin] PATH_INFO: `/v3/roles` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:09:15.815560 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56897625-9e54-46b1-866a-e865bcb039ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:09:15.817156 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56897625-9e54-46b1-866a-e865bcb039ac admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:09:15.827748 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1426/2846] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 1 15:09:15 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 15:09:15.832139 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cb20c9-d7a0-4774-9ce5-f59f304e696e admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:09:15.842320 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cb20c9-d7a0-4774-9ce5-f59f304e696e admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:09:15.847399 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e4c8d5-4a43-4749-a69b-49f5ebe2d2a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:09:15.848040 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e4c8d5-4a43-4749-a69b-49f5ebe2d2a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:09:15.848323 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e4c8d5-4a43-4749-a69b-49f5ebe2d2a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:09:15.848594 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e4c8d5-4a43-4749-a69b-49f5ebe2d2a9 admin admin] PATH_INFO: `/v3/projects/3591aa5c053a40f6afdf04fc159616db/users/a34298f495e44066a76fc2b16de544b7/roles/2a2adb769e55476b82e5014897e0a9c0` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:09:15.864314 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e4c8d5-4a43-4749-a69b-49f5ebe2d2a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3591aa5c053a40f6afdf04fc159616db, user_id=a34298f495e44066a76fc2b16de544b7, role_id=2a2adb769e55476b82e5014897e0a9c0)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:09:15.867027 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e4c8d5-4a43-4749-a69b-49f5ebe2d2a9 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:09:15.879976 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1421/2847] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 1 15:09:15 2022] PUT /identity/v3/projects/3591aa5c053a40f6afdf04fc159616db/users/a34298f495e44066a76fc2b16de544b7/roles/2a2adb769e55476b82e5014897e0a9c0 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 15:09:15.884093 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56897625-9e54-46b1-866a-e865bcb039ac admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:09:15.893995 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56897625-9e54-46b1-866a-e865bcb039ac admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:09:15.913951 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 15:09:15.953774 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01da6fcc-24eb-4582-b245-2fba8c306e86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:09:15.954420 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01da6fcc-24eb-4582-b245-2fba8c306e86 admin admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:09:15.954698 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01da6fcc-24eb-4582-b245-2fba8c306e86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:09:15.954963 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01da6fcc-24eb-4582-b245-2fba8c306e86 admin admin] PATH_INFO: `/v3/roles` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:09:15.955597 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01da6fcc-24eb-4582-b245-2fba8c306e86 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:09:15.957194 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01da6fcc-24eb-4582-b245-2fba8c306e86 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:09:15.966966 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1427/2848] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 1 15:09:15 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 15:09:15.971764 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e4c8d5-4a43-4749-a69b-49f5ebe2d2a9 admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:09:15.977158 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 15:09:15.983521 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e4c8d5-4a43-4749-a69b-49f5ebe2d2a9 admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:09:15.989563 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61aefa6-8e2b-4d96-9406-c0126dde1d59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:09:15.990426 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61aefa6-8e2b-4d96-9406-c0126dde1d59 admin admin] REQUEST_METHOD: `PUT` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:09:15.990798 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61aefa6-8e2b-4d96-9406-c0126dde1d59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:09:15.991181 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61aefa6-8e2b-4d96-9406-c0126dde1d59 admin admin] PATH_INFO: `/v3/projects/3591aa5c053a40f6afdf04fc159616db/users/a34298f495e44066a76fc2b16de544b7/roles/2a2adb769e55476b82e5014897e0a9c0` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:09:15.993207 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61aefa6-8e2b-4d96-9406-c0126dde1d59 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3591aa5c053a40f6afdf04fc159616db, user_id=a34298f495e44066a76fc2b16de544b7, role_id=2a2adb769e55476b82e5014897e0a9c0)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:09:15.995896 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61aefa6-8e2b-4d96-9406-c0126dde1d59 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:09:16.006890 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1422/2849] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 1 15:09:15 2022] PUT /identity/v3/projects/3591aa5c053a40f6afdf04fc159616db/users/a34298f495e44066a76fc2b16de544b7/roles/2a2adb769e55476b82e5014897e0a9c0 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 15:09:16.018476 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52d195e-3797-45fc-8d8f-f4cdb7ce647b None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:09:16.019067 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52d195e-3797-45fc-8d8f-f4cdb7ce647b None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:09:16.019067 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52d195e-3797-45fc-8d8f-f4cdb7ce647b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:09:16.051509 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-f52d195e-3797-45fc-8d8f-f4cdb7ce647b None None] MFA Rules not processed for user `a34298f495e44066a76fc2b16de544b7`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 15:09:16.052738 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-f52d195e-3797-45fc-8d8f-f4cdb7ce647b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:09:16.124490 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1428/2850] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 15:09:16 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 15:09:16.135479 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61aefa6-8e2b-4d96-9406-c0126dde1d59 admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:09:16.137277 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-f61aefa6-8e2b-4d96-9406-c0126dde1d59 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:09:16.194365 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61aefa6-8e2b-4d96-9406-c0126dde1d59 admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:09:16.197926 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5e45c5-fb8c-48a5-a647-608510af9dbc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3b81739e34e4e648f88b6d73fcb5e46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:09:16.198554 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5e45c5-fb8c-48a5-a647-608510af9dbc service cinder] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:09:16.198831 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5e45c5-fb8c-48a5-a647-608510af9dbc service cinder] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:09:16.199097 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5e45c5-fb8c-48a5-a647-608510af9dbc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:09:16.208812 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5e45c5-fb8c-48a5-a647-608510af9dbc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:09:16.210292 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5e45c5-fb8c-48a5-a647-608510af9dbc service cinder] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:09:16.220880 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1423/2851] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 15:09:16 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 15:09:16.249247 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52d195e-3797-45fc-8d8f-f4cdb7ce647b None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:09:16.310255 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52d195e-3797-45fc-8d8f-f4cdb7ce647b None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:09:16.314804 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fff025-3e60-4d55-846d-21a1cc32828f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:09:16.315421 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fff025-3e60-4d55-846d-21a1cc32828f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:09:16.315697 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fff025-3e60-4d55-846d-21a1cc32828f admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:09:16.315967 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fff025-3e60-4d55-846d-21a1cc32828f admin admin] PATH_INFO: `/v3/users/5a704949e57a4e48a05174c607340cc7` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:09:16.317072 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fff025-3e60-4d55-846d-21a1cc32828f admin admin] RBAC: Authorizing `identity:delete_user(user_id=5a704949e57a4e48a05174c607340cc7)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:09:16.319028 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fff025-3e60-4d55-846d-21a1cc32828f admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:09:16.363743 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-38fff025-3e60-4d55-846d-21a1cc32828f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5a704949e57a4e48a05174c607340cc7'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:09:16.365338 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 15:09:16.371382 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-38fff025-3e60-4d55-846d-21a1cc32828f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5a704949e57a4e48a05174c607340cc7'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:09:16.372384 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-38fff025-3e60-4d55-846d-21a1cc32828f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5a704949e57a4e48a05174c607340cc7'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:09:16.383970 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-38fff025-3e60-4d55-846d-21a1cc32828f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5a704949e57a4e48a05174c607340cc7'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:09:16.384550 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-38fff025-3e60-4d55-846d-21a1cc32828f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5a704949e57a4e48a05174c607340cc7'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:09:16.396802 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1429/2852] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Sep 1 15:09:16 2022] DELETE /identity/v3/users/5a704949e57a4e48a05174c607340cc7 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 15:09:16.401208 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6169ec9-a369-49d2-83fa-9b630cc15996 None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:09:16.401488 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6169ec9-a369-49d2-83fa-9b630cc15996 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:09:16.401754 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6169ec9-a369-49d2-83fa-9b630cc15996 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:09:16.420965 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-c6169ec9-a369-49d2-83fa-9b630cc15996 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 15:09:16.422192 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-c6169ec9-a369-49d2-83fa-9b630cc15996 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:09:16.478940 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1424/2853] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 15:09:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 15:09:16.488797 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fff025-3e60-4d55-846d-21a1cc32828f admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:09:16.490219 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-38fff025-3e60-4d55-846d-21a1cc32828f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:09:16.546514 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fff025-3e60-4d55-846d-21a1cc32828f admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:09:16.550141 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20700dc-7e34-4cd2-ae4a-47ffe6746483 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2790149639aa418bbc4e4982e93eda44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:09:16.550762 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20700dc-7e34-4cd2-ae4a-47ffe6746483 service neutron] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:09:16.551035 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20700dc-7e34-4cd2-ae4a-47ffe6746483 service neutron] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:09:16.551301 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20700dc-7e34-4cd2-ae4a-47ffe6746483 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:09:16.559971 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20700dc-7e34-4cd2-ae4a-47ffe6746483 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:09:16.561392 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20700dc-7e34-4cd2-ae4a-47ffe6746483 service neutron] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:09:16.570516 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1430/2854] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 15:09:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 15:09:16.835495 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6169ec9-a369-49d2-83fa-9b630cc15996 None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:09:16.837070 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-c6169ec9-a369-49d2-83fa-9b630cc15996 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:09:16.849223 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6169ec9-a369-49d2-83fa-9b630cc15996 None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:09:16.854129 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a9732b-3260-45f6-8606-4bbe0196217c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:09:16.854791 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a9732b-3260-45f6-8606-4bbe0196217c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:09:16.855066 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a9732b-3260-45f6-8606-4bbe0196217c admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:09:16.855384 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a9732b-3260-45f6-8606-4bbe0196217c admin admin] PATH_INFO: `/v3/projects/46e1fdef6476415f9bd507856d625f28` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:09:16.856630 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a9732b-3260-45f6-8606-4bbe0196217c admin admin] RBAC: Authorizing `identity:delete_project(project_id=46e1fdef6476415f9bd507856d625f28)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:09:16.858617 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a9732b-3260-45f6-8606-4bbe0196217c admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:09:16.905294 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-e8a9732b-3260-45f6-8606-4bbe0196217c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '46e1fdef6476415f9bd507856d625f28', 'request_id': 'req-e8a9732b-3260-45f6-8606-4bbe0196217c'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:09:16.911253 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-e8a9732b-3260-45f6-8606-4bbe0196217c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '46e1fdef6476415f9bd507856d625f28', 'request_id': 'req-e8a9732b-3260-45f6-8606-4bbe0196217c'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:09:16.919272 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-e8a9732b-3260-45f6-8606-4bbe0196217c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '46e1fdef6476415f9bd507856d625f28', 'request_id': 'req-e8a9732b-3260-45f6-8606-4bbe0196217c'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:09:16.929376 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-e8a9732b-3260-45f6-8606-4bbe0196217c admin admin] The token cache is being invalidate because project 46e1fdef6476415f9bd507856d625f28 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81341) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 15:09:16.929662 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-e8a9732b-3260-45f6-8606-4bbe0196217c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:09:16.930298 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-e8a9732b-3260-45f6-8606-4bbe0196217c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:09:16.931753 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1425/2855] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Sep 1 15:09:16 2022] DELETE /identity/v3/projects/46e1fdef6476415f9bd507856d625f28 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 15:09:16.935700 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20700dc-7e34-4cd2-ae4a-47ffe6746483 service neutron] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:09:16.945543 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20700dc-7e34-4cd2-ae4a-47ffe6746483 service neutron] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:09:16.999936 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450959d2-6f9b-46a2-8721-c983ee28ee70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:09:17.001163 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450959d2-6f9b-46a2-8721-c983ee28ee70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:09:17.001441 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450959d2-6f9b-46a2-8721-c983ee28ee70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:09:17.001733 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450959d2-6f9b-46a2-8721-c983ee28ee70 admin admin] PATH_INFO: `/v3/users/a34298f495e44066a76fc2b16de544b7` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:09:17.002843 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450959d2-6f9b-46a2-8721-c983ee28ee70 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a34298f495e44066a76fc2b16de544b7)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:09:17.004764 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450959d2-6f9b-46a2-8721-c983ee28ee70 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:09:17.050691 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-450959d2-6f9b-46a2-8721-c983ee28ee70 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a34298f495e44066a76fc2b16de544b7'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:09:17.057764 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-450959d2-6f9b-46a2-8721-c983ee28ee70 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a34298f495e44066a76fc2b16de544b7'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:09:17.058407 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-450959d2-6f9b-46a2-8721-c983ee28ee70 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a34298f495e44066a76fc2b16de544b7'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:09:17.069778 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:INSERT=2,keystone:DELETE=13,keystone:SELECT=85 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 15:09:17.070996 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-450959d2-6f9b-46a2-8721-c983ee28ee70 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a34298f495e44066a76fc2b16de544b7'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:09:17.071644 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-450959d2-6f9b-46a2-8721-c983ee28ee70 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a34298f495e44066a76fc2b16de544b7'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:09:17.085683 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1431/2856] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Sep 1 15:09:16 2022] DELETE /identity/v3/users/a34298f495e44066a76fc2b16de544b7 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 15:09:17.355456 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a9732b-3260-45f6-8606-4bbe0196217c admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:09:17.356901 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-e8a9732b-3260-45f6-8606-4bbe0196217c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:09:17.371100 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a9732b-3260-45f6-8606-4bbe0196217c admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:09:17.383930 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 15:09:17.431116 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cae9860-8166-4ccd-b947-102df1dc37bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:09:17.431668 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cae9860-8166-4ccd-b947-102df1dc37bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:09:17.431948 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cae9860-8166-4ccd-b947-102df1dc37bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:09:17.432225 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cae9860-8166-4ccd-b947-102df1dc37bb admin admin] PATH_INFO: `/v3/projects/3591aa5c053a40f6afdf04fc159616db` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:09:17.433338 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cae9860-8166-4ccd-b947-102df1dc37bb admin admin] RBAC: Authorizing `identity:delete_project(project_id=3591aa5c053a40f6afdf04fc159616db)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:09:17.435282 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cae9860-8166-4ccd-b947-102df1dc37bb admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:09:17.480585 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-3cae9860-8166-4ccd-b947-102df1dc37bb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3591aa5c053a40f6afdf04fc159616db', 'request_id': 'req-3cae9860-8166-4ccd-b947-102df1dc37bb'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:09:17.487105 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-3cae9860-8166-4ccd-b947-102df1dc37bb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3591aa5c053a40f6afdf04fc159616db', 'request_id': 'req-3cae9860-8166-4ccd-b947-102df1dc37bb'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:09:17.494140 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-3cae9860-8166-4ccd-b947-102df1dc37bb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3591aa5c053a40f6afdf04fc159616db', 'request_id': 'req-3cae9860-8166-4ccd-b947-102df1dc37bb'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:09:17.504246 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-3cae9860-8166-4ccd-b947-102df1dc37bb admin admin] The token cache is being invalidate because project 3591aa5c053a40f6afdf04fc159616db was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81341) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 15:09:17.504531 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-3cae9860-8166-4ccd-b947-102df1dc37bb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:09:17.505148 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-3cae9860-8166-4ccd-b947-102df1dc37bb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:09:17.506702 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1426/2857] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Sep 1 15:09:17 2022] DELETE /identity/v3/projects/3591aa5c053a40f6afdf04fc159616db => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 15:09:17.512380 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3740e2cf-f9be-4fb8-b50d-8a4397c704be None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:09:17.512734 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3740e2cf-f9be-4fb8-b50d-8a4397c704be None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:09:17.513074 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3740e2cf-f9be-4fb8-b50d-8a4397c704be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:09:17.535099 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-3740e2cf-f9be-4fb8-b50d-8a4397c704be None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 15:09:17.536296 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-3740e2cf-f9be-4fb8-b50d-8a4397c704be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:09:17.596388 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1432/2858] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 15:09:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 15:09:17.655527 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d71f27d-1e99-4211-b1ea-2fc8ec2fc921 None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:09:17.655883 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d71f27d-1e99-4211-b1ea-2fc8ec2fc921 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:09:17.656072 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d71f27d-1e99-4211-b1ea-2fc8ec2fc921 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:09:17.677463 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-2d71f27d-1e99-4211-b1ea-2fc8ec2fc921 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 15:09:17.678720 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-2d71f27d-1e99-4211-b1ea-2fc8ec2fc921 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:09:17.686888 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1427/2859] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 15:09:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 15:09:17.690902 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3740e2cf-f9be-4fb8-b50d-8a4397c704be None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:09:17.700500 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3740e2cf-f9be-4fb8-b50d-8a4397c704be None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:09:17.705532 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2468a3b-a8ca-4a01-9a0b-e71d3ff70718 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:09:17.706159 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2468a3b-a8ca-4a01-9a0b-e71d3ff70718 admin admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:09:17.706457 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2468a3b-a8ca-4a01-9a0b-e71d3ff70718 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:09:17.706719 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2468a3b-a8ca-4a01-9a0b-e71d3ff70718 admin admin] PATH_INFO: `/v3/domains` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:09:17.707359 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2468a3b-a8ca-4a01-9a0b-e71d3ff70718 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 15:09:17.707656 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2468a3b-a8ca-4a01-9a0b-e71d3ff70718 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:09:17.709244 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2468a3b-a8ca-4a01-9a0b-e71d3ff70718 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:09:17.721040 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1433/2860] 10.222.0.33 () {58 vars in 1153 bytes} [Thu Sep 1 15:09:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 15:09:17.724974 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d71f27d-1e99-4211-b1ea-2fc8ec2fc921 None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:09:17.734106 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d71f27d-1e99-4211-b1ea-2fc8ec2fc921 None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:09:17.738777 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e01f08-893d-456c-b616-eed01e81f49c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:09:17.739360 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e01f08-893d-456c-b616-eed01e81f49c admin admin] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:09:17.739658 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e01f08-893d-456c-b616-eed01e81f49c admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:09:17.739908 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e01f08-893d-456c-b616-eed01e81f49c admin admin] PATH_INFO: `/v3/projects` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:09:17.740530 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e01f08-893d-456c-b616-eed01e81f49c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:09:17.742448 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e01f08-893d-456c-b616-eed01e81f49c admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:09:17.769098 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1428/2861] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Sep 1 15:09:17 2022] POST /identity/v3/projects => generated 385 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 15:09:17.773427 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2468a3b-a8ca-4a01-9a0b-e71d3ff70718 admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:09:17.785095 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2468a3b-a8ca-4a01-9a0b-e71d3ff70718 admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:09:17.840721 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cb4dfc-acec-45b1-bb77-01343f1e6600 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:09:17.841324 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cb4dfc-acec-45b1-bb77-01343f1e6600 admin admin] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:09:17.841604 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cb4dfc-acec-45b1-bb77-01343f1e6600 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:09:17.841874 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cb4dfc-acec-45b1-bb77-01343f1e6600 admin admin] PATH_INFO: `/v3/users` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:09:17.842534 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08cb4dfc-acec-45b1-bb77-01343f1e6600 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:09:17.845096 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08cb4dfc-acec-45b1-bb77-01343f1e6600 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:09:17.865256 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1434/2862] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Sep 1 15:09:17 2022] POST /identity/v3/users => generated 293 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 15:09:17.869597 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e01f08-893d-456c-b616-eed01e81f49c admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:09:17.879797 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e01f08-893d-456c-b616-eed01e81f49c admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:09:17.884537 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9e09b5-e317-40ee-bb53-b4eb8d905007 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:09:17.885153 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9e09b5-e317-40ee-bb53-b4eb8d905007 admin admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:09:17.885428 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9e09b5-e317-40ee-bb53-b4eb8d905007 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:09:17.885700 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9e09b5-e317-40ee-bb53-b4eb8d905007 admin admin] PATH_INFO: `/v3/roles` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:09:17.886367 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9e09b5-e317-40ee-bb53-b4eb8d905007 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:09:17.887978 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9e09b5-e317-40ee-bb53-b4eb8d905007 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:09:17.898219 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1429/2863] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 1 15:09:17 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 15:09:17.902050 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cb4dfc-acec-45b1-bb77-01343f1e6600 admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:09:17.912404 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cb4dfc-acec-45b1-bb77-01343f1e6600 admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:09:17.917264 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6725439-902d-49ee-8a8e-d38d9f8abd6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:09:17.917850 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6725439-902d-49ee-8a8e-d38d9f8abd6c admin admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:09:17.918131 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6725439-902d-49ee-8a8e-d38d9f8abd6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:09:17.918440 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6725439-902d-49ee-8a8e-d38d9f8abd6c admin admin] PATH_INFO: `/v3/roles` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:09:17.919081 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6725439-902d-49ee-8a8e-d38d9f8abd6c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:09:17.920709 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6725439-902d-49ee-8a8e-d38d9f8abd6c admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:09:17.930363 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1435/2864] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 1 15:09:17 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 15:09:17.934134 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9e09b5-e317-40ee-bb53-b4eb8d905007 admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:09:17.944286 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9e09b5-e317-40ee-bb53-b4eb8d905007 admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:09:17.948858 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8423de-696c-4c74-ba42-b0373eed8401 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:09:17.949490 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8423de-696c-4c74-ba42-b0373eed8401 admin admin] REQUEST_METHOD: `PUT` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:09:17.949768 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8423de-696c-4c74-ba42-b0373eed8401 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:09:17.950036 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8423de-696c-4c74-ba42-b0373eed8401 admin admin] PATH_INFO: `/v3/projects/f90aefe145404a29975f802b8ae7c6a9/users/06e1ada4d1504550a27fd99621604616/roles/5d1feef0e13d4874928bd86258c4e3b2` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:09:17.964727 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8423de-696c-4c74-ba42-b0373eed8401 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f90aefe145404a29975f802b8ae7c6a9, user_id=06e1ada4d1504550a27fd99621604616, role_id=5d1feef0e13d4874928bd86258c4e3b2)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:09:17.967410 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8423de-696c-4c74-ba42-b0373eed8401 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:09:17.981005 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1430/2865] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 1 15:09:17 2022] PUT /identity/v3/projects/f90aefe145404a29975f802b8ae7c6a9/users/06e1ada4d1504550a27fd99621604616/roles/5d1feef0e13d4874928bd86258c4e3b2 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 15:09:17.990783 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7561985d-30b7-4c18-b007-ffa968a53c55 None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:09:17.991068 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7561985d-30b7-4c18-b007-ffa968a53c55 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:09:17.991364 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7561985d-30b7-4c18-b007-ffa968a53c55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:09:18.018908 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-7561985d-30b7-4c18-b007-ffa968a53c55 None None] MFA Rules not processed for user `06e1ada4d1504550a27fd99621604616`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 15:09:18.020105 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-7561985d-30b7-4c18-b007-ffa968a53c55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:09:18.066608 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 15:09:18.090307 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1436/2866] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 15:09:17 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 15:09:18.102125 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8423de-696c-4c74-ba42-b0373eed8401 admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:09:18.103845 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-5a8423de-696c-4c74-ba42-b0373eed8401 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:09:18.149427 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 15:09:18.162432 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8423de-696c-4c74-ba42-b0373eed8401 admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:09:18.166157 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1a4aac-df6e-4255-9adc-3d6fd012daa5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3b81739e34e4e648f88b6d73fcb5e46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:09:18.167032 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1a4aac-df6e-4255-9adc-3d6fd012daa5 service cinder] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:09:18.167323 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1a4aac-df6e-4255-9adc-3d6fd012daa5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:09:18.167576 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1a4aac-df6e-4255-9adc-3d6fd012daa5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:09:18.176088 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1a4aac-df6e-4255-9adc-3d6fd012daa5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:09:18.177593 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1a4aac-df6e-4255-9adc-3d6fd012daa5 service cinder] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:09:18.188253 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1431/2867] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 15:09:18 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 15:09:18.208225 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7561985d-30b7-4c18-b007-ffa968a53c55 None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:09:18.269787 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7561985d-30b7-4c18-b007-ffa968a53c55 None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:09:18.274476 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d2061e-f6fd-477b-b444-560ff244e014 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:09:18.275095 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d2061e-f6fd-477b-b444-560ff244e014 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:09:18.275371 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d2061e-f6fd-477b-b444-560ff244e014 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:09:18.275641 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d2061e-f6fd-477b-b444-560ff244e014 admin admin] PATH_INFO: `/v3/users/06e1ada4d1504550a27fd99621604616` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:09:18.276759 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d2061e-f6fd-477b-b444-560ff244e014 admin admin] RBAC: Authorizing `identity:delete_user(user_id=06e1ada4d1504550a27fd99621604616)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:09:18.278720 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d2061e-f6fd-477b-b444-560ff244e014 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:09:18.323723 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-51d2061e-f6fd-477b-b444-560ff244e014 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '06e1ada4d1504550a27fd99621604616'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:09:18.331342 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-51d2061e-f6fd-477b-b444-560ff244e014 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '06e1ada4d1504550a27fd99621604616'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:09:18.332046 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-51d2061e-f6fd-477b-b444-560ff244e014 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '06e1ada4d1504550a27fd99621604616'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:09:18.343839 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-51d2061e-f6fd-477b-b444-560ff244e014 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '06e1ada4d1504550a27fd99621604616'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:09:18.344488 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-51d2061e-f6fd-477b-b444-560ff244e014 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '06e1ada4d1504550a27fd99621604616'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:09:18.356316 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1437/2868] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Sep 1 15:09:18 2022] DELETE /identity/v3/users/06e1ada4d1504550a27fd99621604616 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 15:09:18.360777 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b9553d-8dae-40d4-9a9e-5f35147785ad None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:09:18.361251 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b9553d-8dae-40d4-9a9e-5f35147785ad None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:09:18.361350 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b9553d-8dae-40d4-9a9e-5f35147785ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:09:18.380500 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-d8b9553d-8dae-40d4-9a9e-5f35147785ad None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 15:09:18.381734 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-d8b9553d-8dae-40d4-9a9e-5f35147785ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:09:18.438809 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1432/2869] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 15:09:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 15:09:18.448575 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d2061e-f6fd-477b-b444-560ff244e014 admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:09:18.449923 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-51d2061e-f6fd-477b-b444-560ff244e014 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:09:18.506522 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d2061e-f6fd-477b-b444-560ff244e014 admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:09:18.510084 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6458c66e-9673-477c-9e91-5c22a7ae295d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2790149639aa418bbc4e4982e93eda44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:09:18.510706 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6458c66e-9673-477c-9e91-5c22a7ae295d service neutron] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:09:18.510981 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6458c66e-9673-477c-9e91-5c22a7ae295d service neutron] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:09:18.511246 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6458c66e-9673-477c-9e91-5c22a7ae295d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:09:18.520499 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6458c66e-9673-477c-9e91-5c22a7ae295d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:09:18.521961 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6458c66e-9673-477c-9e91-5c22a7ae295d service neutron] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:09:18.531229 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1438/2870] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 15:09:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 15:09:18.803403 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b9553d-8dae-40d4-9a9e-5f35147785ad None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:09:18.804864 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-d8b9553d-8dae-40d4-9a9e-5f35147785ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:09:18.818440 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b9553d-8dae-40d4-9a9e-5f35147785ad None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:09:18.824125 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e89171-ded5-4b00-9868-956f57d6e204 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:09:18.824857 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e89171-ded5-4b00-9868-956f57d6e204 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:09:18.825089 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e89171-ded5-4b00-9868-956f57d6e204 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:09:18.825480 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e89171-ded5-4b00-9868-956f57d6e204 admin admin] PATH_INFO: `/v3/projects/f90aefe145404a29975f802b8ae7c6a9` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:09:18.826689 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e89171-ded5-4b00-9868-956f57d6e204 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f90aefe145404a29975f802b8ae7c6a9)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:09:18.828634 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e89171-ded5-4b00-9868-956f57d6e204 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:09:18.875197 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-94e89171-ded5-4b00-9868-956f57d6e204 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f90aefe145404a29975f802b8ae7c6a9', 'request_id': 'req-94e89171-ded5-4b00-9868-956f57d6e204'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:09:18.881344 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-94e89171-ded5-4b00-9868-956f57d6e204 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f90aefe145404a29975f802b8ae7c6a9', 'request_id': 'req-94e89171-ded5-4b00-9868-956f57d6e204'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:09:18.888574 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-94e89171-ded5-4b00-9868-956f57d6e204 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f90aefe145404a29975f802b8ae7c6a9', 'request_id': 'req-94e89171-ded5-4b00-9868-956f57d6e204'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:09:18.898633 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-94e89171-ded5-4b00-9868-956f57d6e204 admin admin] The token cache is being invalidate because project f90aefe145404a29975f802b8ae7c6a9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81341) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 15:09:18.898917 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-94e89171-ded5-4b00-9868-956f57d6e204 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:09:18.899531 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-94e89171-ded5-4b00-9868-956f57d6e204 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:09:18.900989 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1433/2871] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Sep 1 15:09:18 2022] DELETE /identity/v3/projects/f90aefe145404a29975f802b8ae7c6a9 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 15:09:18.905803 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e258fe74-fd64-4987-9d6f-3de8eeb4218a None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:09:18.906094 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e258fe74-fd64-4987-9d6f-3de8eeb4218a None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:09:18.906907 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e258fe74-fd64-4987-9d6f-3de8eeb4218a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:09:18.915419 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 15:09:18.928205 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-e258fe74-fd64-4987-9d6f-3de8eeb4218a None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 15:09:18.930119 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-e258fe74-fd64-4987-9d6f-3de8eeb4218a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:09:18.989101 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1439/2872] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 15:09:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 15:09:18.996757 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce01607-8189-485a-9091-b3b8d8758435 None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:09:18.997044 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce01607-8189-485a-9091-b3b8d8758435 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:09:18.997318 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce01607-8189-485a-9091-b3b8d8758435 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:09:19.016682 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-8ce01607-8189-485a-9091-b3b8d8758435 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 15:09:19.017892 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-8ce01607-8189-485a-9091-b3b8d8758435 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:09:19.025938 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1434/2873] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 15:09:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 15:09:19.030005 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e258fe74-fd64-4987-9d6f-3de8eeb4218a None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:09:19.040073 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e258fe74-fd64-4987-9d6f-3de8eeb4218a None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:09:19.044888 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0249b2b2-086c-4490-a23c-f869d3266dc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:09:19.045475 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0249b2b2-086c-4490-a23c-f869d3266dc1 admin admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:09:19.045751 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0249b2b2-086c-4490-a23c-f869d3266dc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:09:19.046021 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0249b2b2-086c-4490-a23c-f869d3266dc1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:09:19.046700 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0249b2b2-086c-4490-a23c-f869d3266dc1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 15:09:19.047638 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0249b2b2-086c-4490-a23c-f869d3266dc1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:09:19.049230 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0249b2b2-086c-4490-a23c-f869d3266dc1 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:09:19.060548 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1440/2874] 10.222.0.33 () {58 vars in 1153 bytes} [Thu Sep 1 15:09:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 15:09:19.065624 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce01607-8189-485a-9091-b3b8d8758435 None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:09:19.074934 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce01607-8189-485a-9091-b3b8d8758435 None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:09:19.080144 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27448d9f-cd67-424a-939c-216ca969fb8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:09:19.080725 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27448d9f-cd67-424a-939c-216ca969fb8c admin admin] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:09:19.080998 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27448d9f-cd67-424a-939c-216ca969fb8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:09:19.081266 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27448d9f-cd67-424a-939c-216ca969fb8c admin admin] PATH_INFO: `/v3/projects` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:09:19.081885 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27448d9f-cd67-424a-939c-216ca969fb8c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:09:19.083814 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27448d9f-cd67-424a-939c-216ca969fb8c admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:09:19.110141 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1435/2875] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Sep 1 15:09:19 2022] POST /identity/v3/projects => generated 369 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 15:09:19.114189 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0249b2b2-086c-4490-a23c-f869d3266dc1 admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:09:19.124046 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0249b2b2-086c-4490-a23c-f869d3266dc1 admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:09:19.180395 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a267d5-d363-4e7a-93d8-ecd25b20b49e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:09:19.181113 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a267d5-d363-4e7a-93d8-ecd25b20b49e admin admin] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:09:19.181394 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a267d5-d363-4e7a-93d8-ecd25b20b49e admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:09:19.181732 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a267d5-d363-4e7a-93d8-ecd25b20b49e admin admin] PATH_INFO: `/v3/users` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:09:19.182614 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a267d5-d363-4e7a-93d8-ecd25b20b49e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:09:19.184909 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a267d5-d363-4e7a-93d8-ecd25b20b49e admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:09:19.206425 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1441/2876] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Sep 1 15:09:19 2022] POST /identity/v3/users => generated 285 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 15:09:19.210506 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27448d9f-cd67-424a-939c-216ca969fb8c admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:09:19.220576 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27448d9f-cd67-424a-939c-216ca969fb8c admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:09:19.225056 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ac70c3-ed4e-4b56-8438-79b409dd65ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:09:19.225635 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ac70c3-ed4e-4b56-8438-79b409dd65ca admin admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:09:19.225909 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ac70c3-ed4e-4b56-8438-79b409dd65ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:09:19.226179 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ac70c3-ed4e-4b56-8438-79b409dd65ca admin admin] PATH_INFO: `/v3/roles` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:09:19.226842 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ac70c3-ed4e-4b56-8438-79b409dd65ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:09:19.228442 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ac70c3-ed4e-4b56-8438-79b409dd65ca admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:09:19.238863 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1436/2877] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 1 15:09:19 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 15:09:19.242840 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a267d5-d363-4e7a-93d8-ecd25b20b49e admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:09:19.253527 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a267d5-d363-4e7a-93d8-ecd25b20b49e admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:09:19.258378 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7093c47c-c948-480c-b7ee-2cd0659160ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:09:19.258994 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7093c47c-c948-480c-b7ee-2cd0659160ca admin admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:09:19.259268 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7093c47c-c948-480c-b7ee-2cd0659160ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:09:19.259542 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7093c47c-c948-480c-b7ee-2cd0659160ca admin admin] PATH_INFO: `/v3/roles` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:09:19.260203 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7093c47c-c948-480c-b7ee-2cd0659160ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:09:19.261800 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7093c47c-c948-480c-b7ee-2cd0659160ca admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:09:19.273127 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1442/2878] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 1 15:09:19 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 15:09:19.277040 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ac70c3-ed4e-4b56-8438-79b409dd65ca admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:09:19.286766 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ac70c3-ed4e-4b56-8438-79b409dd65ca admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:09:19.291273 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df80fa63-781c-4ee4-b442-53b9860af715 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:09:19.292153 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df80fa63-781c-4ee4-b442-53b9860af715 admin admin] REQUEST_METHOD: `PUT` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:09:19.292425 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df80fa63-781c-4ee4-b442-53b9860af715 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:09:19.292690 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df80fa63-781c-4ee4-b442-53b9860af715 admin admin] PATH_INFO: `/v3/projects/44c6c57d15724c7f932c4dcaf1d08d1c/users/06ad3c658f9c4fdda949b2823e917ed0/roles/5d1feef0e13d4874928bd86258c4e3b2` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:09:19.307324 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df80fa63-781c-4ee4-b442-53b9860af715 admin admin] RBAC: Authorizing `identity:create_grant(project_id=44c6c57d15724c7f932c4dcaf1d08d1c, user_id=06ad3c658f9c4fdda949b2823e917ed0, role_id=5d1feef0e13d4874928bd86258c4e3b2)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:09:19.309936 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df80fa63-781c-4ee4-b442-53b9860af715 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:09:19.322975 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1437/2879] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 1 15:09:19 2022] PUT /identity/v3/projects/44c6c57d15724c7f932c4dcaf1d08d1c/users/06ad3c658f9c4fdda949b2823e917ed0/roles/5d1feef0e13d4874928bd86258c4e3b2 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 15:09:19.334162 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b6a5b5-6f9b-47d2-8ec0-a8538156af72 None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:09:19.334487 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b6a5b5-6f9b-47d2-8ec0-a8538156af72 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:09:19.334757 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b6a5b5-6f9b-47d2-8ec0-a8538156af72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:09:19.362053 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-a0b6a5b5-6f9b-47d2-8ec0-a8538156af72 None None] MFA Rules not processed for user `06ad3c658f9c4fdda949b2823e917ed0`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 15:09:19.363296 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-a0b6a5b5-6f9b-47d2-8ec0-a8538156af72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:09:19.423542 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 15:09:19.427646 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1443/2880] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 15:09:19 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 15:09:19.440292 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df80fa63-781c-4ee4-b442-53b9860af715 admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:09:19.442115 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-df80fa63-781c-4ee4-b442-53b9860af715 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:09:19.446845 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 15:09:19.520607 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df80fa63-781c-4ee4-b442-53b9860af715 admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:09:19.524143 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ceb15b7-44a0-4dd2-8b6a-9e60c81f5b4a service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3069f72ce98049b99c9d98229a62bb51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:09:19.524731 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ceb15b7-44a0-4dd2-8b6a-9e60c81f5b4a service glance] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:09:19.525005 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ceb15b7-44a0-4dd2-8b6a-9e60c81f5b4a service glance] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:09:19.525273 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ceb15b7-44a0-4dd2-8b6a-9e60c81f5b4a service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:09:19.533329 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ceb15b7-44a0-4dd2-8b6a-9e60c81f5b4a service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:09:19.534827 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ceb15b7-44a0-4dd2-8b6a-9e60c81f5b4a service glance] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:09:19.543568 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1438/2881] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 15:09:19 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 15:09:29.539440 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=33 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 15:09:46.226565 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b6a5b5-6f9b-47d2-8ec0-a8538156af72 None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:09:46.290432 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b6a5b5-6f9b-47d2-8ec0-a8538156af72 None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:09:46.295837 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60713cfa-2ab8-4abc-9b18-f0388e551e4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:09:46.296490 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60713cfa-2ab8-4abc-9b18-f0388e551e4c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:09:46.296769 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60713cfa-2ab8-4abc-9b18-f0388e551e4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:09:46.297037 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60713cfa-2ab8-4abc-9b18-f0388e551e4c admin admin] PATH_INFO: `/v3/users/06ad3c658f9c4fdda949b2823e917ed0` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:09:46.298160 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60713cfa-2ab8-4abc-9b18-f0388e551e4c admin admin] RBAC: Authorizing `identity:delete_user(user_id=06ad3c658f9c4fdda949b2823e917ed0)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:09:46.300166 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60713cfa-2ab8-4abc-9b18-f0388e551e4c admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:09:46.346618 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-60713cfa-2ab8-4abc-9b18-f0388e551e4c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '06ad3c658f9c4fdda949b2823e917ed0'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:09:46.354868 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-60713cfa-2ab8-4abc-9b18-f0388e551e4c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '06ad3c658f9c4fdda949b2823e917ed0'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:09:46.355600 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-60713cfa-2ab8-4abc-9b18-f0388e551e4c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '06ad3c658f9c4fdda949b2823e917ed0'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:09:46.368484 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-60713cfa-2ab8-4abc-9b18-f0388e551e4c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '06ad3c658f9c4fdda949b2823e917ed0'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:09:46.369945 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-60713cfa-2ab8-4abc-9b18-f0388e551e4c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '06ad3c658f9c4fdda949b2823e917ed0'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:09:46.382301 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1444/2882] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Sep 1 15:09:46 2022] DELETE /identity/v3/users/06ad3c658f9c4fdda949b2823e917ed0 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 15:09:46.386920 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7ef0b9-d96b-46c7-a16e-f6ee0b393175 None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:09:46.387420 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7ef0b9-d96b-46c7-a16e-f6ee0b393175 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:09:46.387541 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7ef0b9-d96b-46c7-a16e-f6ee0b393175 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:09:46.406218 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-1d7ef0b9-d96b-46c7-a16e-f6ee0b393175 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 15:09:46.407448 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-1d7ef0b9-d96b-46c7-a16e-f6ee0b393175 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:09:46.465125 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1439/2883] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 15:09:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 15:09:46.475104 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60713cfa-2ab8-4abc-9b18-f0388e551e4c admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:09:46.476636 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-60713cfa-2ab8-4abc-9b18-f0388e551e4c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:09:46.522625 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7ef0b9-d96b-46c7-a16e-f6ee0b393175 None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:09:46.524552 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-1d7ef0b9-d96b-46c7-a16e-f6ee0b393175 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:09:46.540176 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60713cfa-2ab8-4abc-9b18-f0388e551e4c admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:09:46.545068 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093db2aa-415e-4794-929b-a64328b7d120 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2790149639aa418bbc4e4982e93eda44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:09:46.545868 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093db2aa-415e-4794-929b-a64328b7d120 service neutron] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:09:46.546298 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093db2aa-415e-4794-929b-a64328b7d120 service neutron] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:09:46.546684 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093db2aa-415e-4794-929b-a64328b7d120 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:09:46.561548 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093db2aa-415e-4794-929b-a64328b7d120 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:09:46.564071 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093db2aa-415e-4794-929b-a64328b7d120 service neutron] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:09:46.574828 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1445/2884] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 15:09:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 15:09:46.589412 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7ef0b9-d96b-46c7-a16e-f6ee0b393175 None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:09:46.593474 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0589cb-ed95-4416-87ce-0bb80673069a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2790149639aa418bbc4e4982e93eda44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:09:46.594267 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0589cb-ed95-4416-87ce-0bb80673069a service neutron] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:09:46.594676 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0589cb-ed95-4416-87ce-0bb80673069a service neutron] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:09:46.595121 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0589cb-ed95-4416-87ce-0bb80673069a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:09:46.596979 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-bd0589cb-ed95-4416-87ce-0bb80673069a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:09:46.609613 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0589cb-ed95-4416-87ce-0bb80673069a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:09:46.611184 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0589cb-ed95-4416-87ce-0bb80673069a service neutron] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:09:46.620700 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1440/2885] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 15:09:46 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 15:09:46.880186 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093db2aa-415e-4794-929b-a64328b7d120 service neutron] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:09:46.881646 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-093db2aa-415e-4794-929b-a64328b7d120 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:09:46.894653 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093db2aa-415e-4794-929b-a64328b7d120 service neutron] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:09:46.899553 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f1f671-75a0-4ba5-9703-dcb7dabcef25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:09:46.900871 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f1f671-75a0-4ba5-9703-dcb7dabcef25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:09:46.901149 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f1f671-75a0-4ba5-9703-dcb7dabcef25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:09:46.901415 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f1f671-75a0-4ba5-9703-dcb7dabcef25 admin admin] PATH_INFO: `/v3/projects/44c6c57d15724c7f932c4dcaf1d08d1c` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:09:46.902613 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f1f671-75a0-4ba5-9703-dcb7dabcef25 admin admin] RBAC: Authorizing `identity:delete_project(project_id=44c6c57d15724c7f932c4dcaf1d08d1c)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:09:46.904534 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f1f671-75a0-4ba5-9703-dcb7dabcef25 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:09:46.925578 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 15:09:46.957304 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-51f1f671-75a0-4ba5-9703-dcb7dabcef25 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '44c6c57d15724c7f932c4dcaf1d08d1c', 'request_id': 'req-51f1f671-75a0-4ba5-9703-dcb7dabcef25'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:09:46.964474 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-51f1f671-75a0-4ba5-9703-dcb7dabcef25 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '44c6c57d15724c7f932c4dcaf1d08d1c', 'request_id': 'req-51f1f671-75a0-4ba5-9703-dcb7dabcef25'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:09:46.973215 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-51f1f671-75a0-4ba5-9703-dcb7dabcef25 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '44c6c57d15724c7f932c4dcaf1d08d1c', 'request_id': 'req-51f1f671-75a0-4ba5-9703-dcb7dabcef25'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:09:46.987146 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-51f1f671-75a0-4ba5-9703-dcb7dabcef25 admin admin] The token cache is being invalidate because project 44c6c57d15724c7f932c4dcaf1d08d1c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81342) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 15:09:46.987576 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-51f1f671-75a0-4ba5-9703-dcb7dabcef25 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:09:46.988503 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-51f1f671-75a0-4ba5-9703-dcb7dabcef25 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:09:46.990688 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1446/2886] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Sep 1 15:09:46 2022] DELETE /identity/v3/projects/44c6c57d15724c7f932c4dcaf1d08d1c => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 15:09:46.996214 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87b6d71-0ba9-4d6e-abe6-2f67c6af8868 None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:09:46.996674 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87b6d71-0ba9-4d6e-abe6-2f67c6af8868 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:09:46.997083 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87b6d71-0ba9-4d6e-abe6-2f67c6af8868 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:09:47.023828 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-c87b6d71-0ba9-4d6e-abe6-2f67c6af8868 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 15:09:47.025609 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-c87b6d71-0ba9-4d6e-abe6-2f67c6af8868 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:09:47.091761 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1441/2887] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 15:09:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 15:09:47.099525 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84727951-c358-417e-be7a-764e67b7ddfb None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:09:47.099810 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84727951-c358-417e-be7a-764e67b7ddfb None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:09:47.100079 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84727951-c358-417e-be7a-764e67b7ddfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:09:47.120864 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-84727951-c358-417e-be7a-764e67b7ddfb None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 15:09:47.122362 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-84727951-c358-417e-be7a-764e67b7ddfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:09:47.131153 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1447/2888] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 15:09:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 15:09:47.135988 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87b6d71-0ba9-4d6e-abe6-2f67c6af8868 None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:09:47.146608 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87b6d71-0ba9-4d6e-abe6-2f67c6af8868 None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:09:47.153512 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb13ed51-e54a-4e43-bd43-d74064c87bb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:09:47.154219 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb13ed51-e54a-4e43-bd43-d74064c87bb8 admin admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:09:47.154716 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb13ed51-e54a-4e43-bd43-d74064c87bb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:09:47.155101 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb13ed51-e54a-4e43-bd43-d74064c87bb8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:09:47.155861 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb13ed51-e54a-4e43-bd43-d74064c87bb8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81341) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 15:09:47.156260 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb13ed51-e54a-4e43-bd43-d74064c87bb8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:09:47.158137 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb13ed51-e54a-4e43-bd43-d74064c87bb8 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:09:47.172485 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1442/2889] 10.222.0.33 () {58 vars in 1153 bytes} [Thu Sep 1 15:09:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 15:09:47.177451 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84727951-c358-417e-be7a-764e67b7ddfb None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:09:47.188621 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84727951-c358-417e-be7a-764e67b7ddfb None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:09:47.193377 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1549b793-2503-4768-9af2-8d7893096949 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:09:47.193975 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1549b793-2503-4768-9af2-8d7893096949 admin admin] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:09:47.194273 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1549b793-2503-4768-9af2-8d7893096949 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:09:47.194541 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1549b793-2503-4768-9af2-8d7893096949 admin admin] PATH_INFO: `/v3/projects` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:09:47.195576 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1549b793-2503-4768-9af2-8d7893096949 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:09:47.197980 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1549b793-2503-4768-9af2-8d7893096949 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:09:47.230705 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1448/2890] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Sep 1 15:09:47 2022] POST /identity/v3/projects => generated 375 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 15:09:47.232927 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb13ed51-e54a-4e43-bd43-d74064c87bb8 admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:09:47.245560 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb13ed51-e54a-4e43-bd43-d74064c87bb8 admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:09:47.294225 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=100 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 15:09:47.325200 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc48d5cb-216c-456b-8433-b43e29a53672 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:09:47.325841 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc48d5cb-216c-456b-8433-b43e29a53672 admin admin] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:09:47.326120 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc48d5cb-216c-456b-8433-b43e29a53672 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:09:47.326422 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc48d5cb-216c-456b-8433-b43e29a53672 admin admin] PATH_INFO: `/v3/users` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:09:47.327054 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc48d5cb-216c-456b-8433-b43e29a53672 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:09:47.329018 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc48d5cb-216c-456b-8433-b43e29a53672 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:09:47.350807 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1443/2891] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Sep 1 15:09:47 2022] POST /identity/v3/users => generated 288 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 15:09:47.354926 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1549b793-2503-4768-9af2-8d7893096949 admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:09:47.365469 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1549b793-2503-4768-9af2-8d7893096949 admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:09:47.370271 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a9b687-5f30-4748-8a02-a1a0b0b3d63d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:09:47.370896 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a9b687-5f30-4748-8a02-a1a0b0b3d63d admin admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:09:47.371175 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a9b687-5f30-4748-8a02-a1a0b0b3d63d admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:09:47.371442 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a9b687-5f30-4748-8a02-a1a0b0b3d63d admin admin] PATH_INFO: `/v3/roles` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:09:47.372133 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a9b687-5f30-4748-8a02-a1a0b0b3d63d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:09:47.373754 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a9b687-5f30-4748-8a02-a1a0b0b3d63d admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:09:47.385375 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1449/2892] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 1 15:09:47 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 15:09:47.389267 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc48d5cb-216c-456b-8433-b43e29a53672 admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:09:47.400665 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc48d5cb-216c-456b-8433-b43e29a53672 admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:09:47.405829 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15491003-58c4-42fb-928f-d15c27ac3730 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:09:47.406519 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15491003-58c4-42fb-928f-d15c27ac3730 admin admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:09:47.406798 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15491003-58c4-42fb-928f-d15c27ac3730 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:09:47.407072 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15491003-58c4-42fb-928f-d15c27ac3730 admin admin] PATH_INFO: `/v3/roles` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:09:47.407729 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15491003-58c4-42fb-928f-d15c27ac3730 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:09:47.409410 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15491003-58c4-42fb-928f-d15c27ac3730 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:09:47.420945 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1444/2893] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 1 15:09:47 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 15:09:47.425131 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a9b687-5f30-4748-8a02-a1a0b0b3d63d admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:09:47.435604 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a9b687-5f30-4748-8a02-a1a0b0b3d63d admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:09:47.440441 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e7a128-645f-4a76-939d-0965261011dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:09:47.441099 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e7a128-645f-4a76-939d-0965261011dd admin admin] REQUEST_METHOD: `PUT` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:09:47.441380 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e7a128-645f-4a76-939d-0965261011dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:09:47.441654 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e7a128-645f-4a76-939d-0965261011dd admin admin] PATH_INFO: `/v3/projects/d429d17d00c24f22b74b56446fcaf35a/users/2c138f1d6eb441228a9770f1c8c36778/roles/5d1feef0e13d4874928bd86258c4e3b2` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:09:47.468094 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e7a128-645f-4a76-939d-0965261011dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=d429d17d00c24f22b74b56446fcaf35a, user_id=2c138f1d6eb441228a9770f1c8c36778, role_id=5d1feef0e13d4874928bd86258c4e3b2)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:09:47.470830 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e7a128-645f-4a76-939d-0965261011dd admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:09:47.485204 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1450/2894] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 1 15:09:47 2022] PUT /identity/v3/projects/d429d17d00c24f22b74b56446fcaf35a/users/2c138f1d6eb441228a9770f1c8c36778/roles/5d1feef0e13d4874928bd86258c4e3b2 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 15:09:47.497042 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff47e28f-064a-4e4b-a12e-9747981f6d5d None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:09:47.497326 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff47e28f-064a-4e4b-a12e-9747981f6d5d None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:09:47.497606 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff47e28f-064a-4e4b-a12e-9747981f6d5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:09:47.525943 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-ff47e28f-064a-4e4b-a12e-9747981f6d5d None None] MFA Rules not processed for user `2c138f1d6eb441228a9770f1c8c36778`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 15:09:47.527228 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-ff47e28f-064a-4e4b-a12e-9747981f6d5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:09:47.595481 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1445/2895] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 15:09:47 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 15:09:47.606649 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e7a128-645f-4a76-939d-0965261011dd admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:09:47.608484 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-16e7a128-645f-4a76-939d-0965261011dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:09:47.673773 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e7a128-645f-4a76-939d-0965261011dd admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:09:47.677461 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199fc095-5d7a-4fdd-ad8b-016c2cb4f89d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3b81739e34e4e648f88b6d73fcb5e46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:09:47.678075 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199fc095-5d7a-4fdd-ad8b-016c2cb4f89d service cinder] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:09:47.678393 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199fc095-5d7a-4fdd-ad8b-016c2cb4f89d service cinder] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:09:47.678672 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199fc095-5d7a-4fdd-ad8b-016c2cb4f89d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:09:47.687863 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199fc095-5d7a-4fdd-ad8b-016c2cb4f89d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:09:47.689304 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199fc095-5d7a-4fdd-ad8b-016c2cb4f89d service cinder] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:09:47.699794 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1451/2896] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 15:09:47 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 15:09:48.443717 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff47e28f-064a-4e4b-a12e-9747981f6d5d None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:09:48.445383 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-ff47e28f-064a-4e4b-a12e-9747981f6d5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:09:48.542963 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff47e28f-064a-4e4b-a12e-9747981f6d5d None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:09:48.548532 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2f01ae-ea2e-421a-b050-4f5ea7a544f5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4858bb8621854ec8aba836e7d9050498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:09:48.549314 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2f01ae-ea2e-421a-b050-4f5ea7a544f5 service nova] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:09:48.549664 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2f01ae-ea2e-421a-b050-4f5ea7a544f5 service nova] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:09:48.550010 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2f01ae-ea2e-421a-b050-4f5ea7a544f5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:09:48.552106 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-2e2f01ae-ea2e-421a-b050-4f5ea7a544f5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:09:48.567507 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2f01ae-ea2e-421a-b050-4f5ea7a544f5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:09:48.569397 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2f01ae-ea2e-421a-b050-4f5ea7a544f5 service nova] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:09:48.598431 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1446/2897] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 15:09:48 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 15:09:49.081331 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0e9884-8793-4aed-9231-dc534b051817 None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:09:49.081913 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0e9884-8793-4aed-9231-dc534b051817 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:09:49.082041 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0e9884-8793-4aed-9231-dc534b051817 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:09:49.101894 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-2a0e9884-8793-4aed-9231-dc534b051817 None None] MFA Rules not processed for user `2c138f1d6eb441228a9770f1c8c36778`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 15:09:49.103895 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-2a0e9884-8793-4aed-9231-dc534b051817 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:09:49.111126 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1452/2898] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 15:09:49 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 15:09:49.585804 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2f01ae-ea2e-421a-b050-4f5ea7a544f5 service nova] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:09:49.587282 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-2e2f01ae-ea2e-421a-b050-4f5ea7a544f5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:09:49.615871 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 15:09:49.660922 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2f01ae-ea2e-421a-b050-4f5ea7a544f5 service nova] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:09:49.667659 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ceba11-24c2-482f-9f3a-bf10be4caea8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f312c8dcabe48ef9ea453364ade38ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:09:49.668420 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ceba11-24c2-482f-9f3a-bf10be4caea8 service placement] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:09:49.668695 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ceba11-24c2-482f-9f3a-bf10be4caea8 service placement] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:09:49.668964 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ceba11-24c2-482f-9f3a-bf10be4caea8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:09:49.670828 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-96ceba11-24c2-482f-9f3a-bf10be4caea8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:09:49.683184 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ceba11-24c2-482f-9f3a-bf10be4caea8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:09:49.684767 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ceba11-24c2-482f-9f3a-bf10be4caea8 service placement] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:09:49.694871 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1447/2899] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 15:09:49 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 15:09:52.260162 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0e9884-8793-4aed-9231-dc534b051817 None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:09:52.261641 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-2a0e9884-8793-4aed-9231-dc534b051817 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:09:52.279636 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0e9884-8793-4aed-9231-dc534b051817 None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:09:52.285590 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335605f1-10e1-4585-98f6-934675a04202 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4858bb8621854ec8aba836e7d9050498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:09:52.286368 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335605f1-10e1-4585-98f6-934675a04202 service nova] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:09:52.286651 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335605f1-10e1-4585-98f6-934675a04202 service nova] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:09:52.286923 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335605f1-10e1-4585-98f6-934675a04202 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:09:52.288680 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-335605f1-10e1-4585-98f6-934675a04202 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:09:52.301864 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335605f1-10e1-4585-98f6-934675a04202 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:09:52.303500 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335605f1-10e1-4585-98f6-934675a04202 service nova] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:09:52.315070 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1453/2900] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 15:09:52 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 15:09:55.868652 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ceba11-24c2-482f-9f3a-bf10be4caea8 service placement] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:09:55.881880 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ceba11-24c2-482f-9f3a-bf10be4caea8 service placement] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:09:55.888143 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036318b1-3a34-4faa-89b8-2947c6c3142f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4858bb8621854ec8aba836e7d9050498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:09:55.888750 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036318b1-3a34-4faa-89b8-2947c6c3142f service nova] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:09:55.889028 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036318b1-3a34-4faa-89b8-2947c6c3142f service nova] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:09:55.889299 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036318b1-3a34-4faa-89b8-2947c6c3142f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:09:55.891216 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-036318b1-3a34-4faa-89b8-2947c6c3142f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:09:55.905567 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036318b1-3a34-4faa-89b8-2947c6c3142f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:09:55.907244 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036318b1-3a34-4faa-89b8-2947c6c3142f service nova] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:09:55.920601 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1448/2901] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 15:09:55 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 15:10:02.310383 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=79,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 15:10:05.507393 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335605f1-10e1-4585-98f6-934675a04202 service nova] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:10:05.509680 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-335605f1-10e1-4585-98f6-934675a04202 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:10:05.569599 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335605f1-10e1-4585-98f6-934675a04202 service nova] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:10:05.573251 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d481d67-8dd3-42d1-b005-4750bfde9e0f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2790149639aa418bbc4e4982e93eda44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:10:05.573858 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d481d67-8dd3-42d1-b005-4750bfde9e0f service neutron] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:10:05.574139 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d481d67-8dd3-42d1-b005-4750bfde9e0f service neutron] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:10:05.574433 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d481d67-8dd3-42d1-b005-4750bfde9e0f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:10:05.576150 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-6d481d67-8dd3-42d1-b005-4750bfde9e0f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:10:05.637732 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d481d67-8dd3-42d1-b005-4750bfde9e0f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:10:05.639198 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d481d67-8dd3-42d1-b005-4750bfde9e0f service neutron] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:10:05.648567 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1454/2902] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 15:10:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 15:10:05.914520 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=27 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 15:10:07.362985 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70bbc20-21eb-474b-9fe0-f660f620d7b1 None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:10:07.363925 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70bbc20-21eb-474b-9fe0-f660f620d7b1 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:10:07.364530 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70bbc20-21eb-474b-9fe0-f660f620d7b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:10:07.385752 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-b70bbc20-21eb-474b-9fe0-f660f620d7b1 None None] MFA Rules not processed for user `24b6f275ffb04ffeae3c2097bfd040f5`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 15:10:07.387428 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-b70bbc20-21eb-474b-9fe0-f660f620d7b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:10:07.450559 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1449/2903] 10.222.0.33 () {58 vars in 1009 bytes} [Thu Sep 1 15:10:07 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 15:10:07.510464 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d481d67-8dd3-42d1-b005-4750bfde9e0f service neutron] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:10:07.512479 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-6d481d67-8dd3-42d1-b005-4750bfde9e0f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:10:07.593708 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d481d67-8dd3-42d1-b005-4750bfde9e0f service neutron] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:10:07.600502 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256ed40e-129b-4b78-9de0-a7a07eb778af service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73a5bda5a6ea46e9863ccafd57dee339', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:10:07.601122 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256ed40e-129b-4b78-9de0-a7a07eb778af service swift] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:10:07.601515 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256ed40e-129b-4b78-9de0-a7a07eb778af service swift] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:10:07.601701 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256ed40e-129b-4b78-9de0-a7a07eb778af service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:10:07.611030 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256ed40e-129b-4b78-9de0-a7a07eb778af service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:10:07.613017 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256ed40e-129b-4b78-9de0-a7a07eb778af service swift] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:10:07.623294 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1455/2904] 10.222.0.33 () {60 vars in 1384 bytes} [Thu Sep 1 15:10:07 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 01 15:10:08.193816 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb51e07-4f2f-419b-bca8-b071330c4389 None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:10:08.194107 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb51e07-4f2f-419b-bca8-b071330c4389 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:10:08.194440 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb51e07-4f2f-419b-bca8-b071330c4389 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:10:08.216160 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-5fb51e07-4f2f-419b-bca8-b071330c4389 None None] MFA Rules not processed for user `24b6f275ffb04ffeae3c2097bfd040f5`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 15:10:08.217394 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-5fb51e07-4f2f-419b-bca8-b071330c4389 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:10:08.225275 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1450/2905] 10.222.0.33 () {58 vars in 967 bytes} [Thu Sep 1 15:10:08 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 15:10:08.239823 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256ed40e-129b-4b78-9de0-a7a07eb778af service swift] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:10:08.251622 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256ed40e-129b-4b78-9de0-a7a07eb778af service swift] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:10:08.263102 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68760f74-3dc0-42cb-9efb-71d87e6da602 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73a5bda5a6ea46e9863ccafd57dee339', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:10:08.264870 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68760f74-3dc0-42cb-9efb-71d87e6da602 service swift] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:10:08.265157 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68760f74-3dc0-42cb-9efb-71d87e6da602 service swift] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:10:08.265425 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68760f74-3dc0-42cb-9efb-71d87e6da602 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:10:08.281667 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68760f74-3dc0-42cb-9efb-71d87e6da602 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:10:08.283513 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68760f74-3dc0-42cb-9efb-71d87e6da602 service swift] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:10:08.295586 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1456/2906] 10.222.0.33 () {60 vars in 1384 bytes} [Thu Sep 1 15:10:08 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 01 15:10:08.553702 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4b24a3-fda4-40d1-a437-4d6dd4dc2ec5 None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:10:08.554189 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4b24a3-fda4-40d1-a437-4d6dd4dc2ec5 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:10:08.554321 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4b24a3-fda4-40d1-a437-4d6dd4dc2ec5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:10:08.573416 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-1f4b24a3-fda4-40d1-a437-4d6dd4dc2ec5 None None] MFA Rules not processed for user `24b6f275ffb04ffeae3c2097bfd040f5`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 15:10:08.574592 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-1f4b24a3-fda4-40d1-a437-4d6dd4dc2ec5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:10:08.582079 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1451/2907] 10.222.0.33 () {58 vars in 1009 bytes} [Thu Sep 1 15:10:08 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 15:10:08.594950 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68760f74-3dc0-42cb-9efb-71d87e6da602 service swift] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:10:08.606003 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68760f74-3dc0-42cb-9efb-71d87e6da602 service swift] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:10:08.611082 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e947b03-dc6f-4715-afb1-08db6f1f8135 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73a5bda5a6ea46e9863ccafd57dee339', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:10:08.611755 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e947b03-dc6f-4715-afb1-08db6f1f8135 service swift] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:10:08.612102 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e947b03-dc6f-4715-afb1-08db6f1f8135 service swift] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:10:08.612441 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e947b03-dc6f-4715-afb1-08db6f1f8135 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:10:08.620669 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e947b03-dc6f-4715-afb1-08db6f1f8135 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:10:08.622426 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e947b03-dc6f-4715-afb1-08db6f1f8135 service swift] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:10:08.632247 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1457/2908] 10.222.0.33 () {60 vars in 1384 bytes} [Thu Sep 1 15:10:08 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 01 15:10:09.060702 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5cee17-a235-49b0-bb93-86d0ea510c76 None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:10:09.061079 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5cee17-a235-49b0-bb93-86d0ea510c76 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:10:09.061421 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5cee17-a235-49b0-bb93-86d0ea510c76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:10:09.082630 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-7d5cee17-a235-49b0-bb93-86d0ea510c76 None None] MFA Rules not processed for user `24b6f275ffb04ffeae3c2097bfd040f5`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 15:10:09.083940 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-7d5cee17-a235-49b0-bb93-86d0ea510c76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:10:09.092002 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1452/2909] 10.222.0.33 () {58 vars in 967 bytes} [Thu Sep 1 15:10:09 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 15:10:09.104378 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e947b03-dc6f-4715-afb1-08db6f1f8135 service swift] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:10:09.115418 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e947b03-dc6f-4715-afb1-08db6f1f8135 service swift] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:10:09.121746 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e6fffe-92f6-4cc2-938b-57ef0c5d5cba service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73a5bda5a6ea46e9863ccafd57dee339', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:10:09.122501 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e6fffe-92f6-4cc2-938b-57ef0c5d5cba service swift] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:10:09.122851 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e6fffe-92f6-4cc2-938b-57ef0c5d5cba service swift] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:10:09.123192 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e6fffe-92f6-4cc2-938b-57ef0c5d5cba service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:10:09.132309 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e6fffe-92f6-4cc2-938b-57ef0c5d5cba service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:10:09.134557 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e6fffe-92f6-4cc2-938b-57ef0c5d5cba service swift] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:10:09.139261 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=100 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 15:10:09.143859 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1458/2910] 10.222.0.33 () {60 vars in 1384 bytes} [Thu Sep 1 15:10:09 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 01 15:10:11.309448 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5cee17-a235-49b0-bb93-86d0ea510c76 None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:10:11.320520 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5cee17-a235-49b0-bb93-86d0ea510c76 None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:10:11.325188 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdfb179-03cb-401f-8e52-d61da176ae55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:10:11.325807 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdfb179-03cb-401f-8e52-d61da176ae55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:10:11.326084 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdfb179-03cb-401f-8e52-d61da176ae55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:10:11.326390 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdfb179-03cb-401f-8e52-d61da176ae55 admin admin] PATH_INFO: `/v3/users/2c138f1d6eb441228a9770f1c8c36778` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:10:11.327928 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cdfb179-03cb-401f-8e52-d61da176ae55 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2c138f1d6eb441228a9770f1c8c36778)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:10:11.329976 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cdfb179-03cb-401f-8e52-d61da176ae55 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:10:11.382427 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-2cdfb179-03cb-401f-8e52-d61da176ae55 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2c138f1d6eb441228a9770f1c8c36778'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:10:11.392152 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-2cdfb179-03cb-401f-8e52-d61da176ae55 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2c138f1d6eb441228a9770f1c8c36778'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:10:11.392800 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-2cdfb179-03cb-401f-8e52-d61da176ae55 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2c138f1d6eb441228a9770f1c8c36778'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:10:11.405132 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-2cdfb179-03cb-401f-8e52-d61da176ae55 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2c138f1d6eb441228a9770f1c8c36778'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:10:11.405717 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-2cdfb179-03cb-401f-8e52-d61da176ae55 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2c138f1d6eb441228a9770f1c8c36778'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:10:11.419539 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1453/2911] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Sep 1 15:10:11 2022] DELETE /identity/v3/users/2c138f1d6eb441228a9770f1c8c36778 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 15:10:11.424027 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667f8ce9-1bcf-4c36-830e-fed3f514d0e0 None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:10:11.424876 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667f8ce9-1bcf-4c36-830e-fed3f514d0e0 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:10:11.425264 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667f8ce9-1bcf-4c36-830e-fed3f514d0e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:10:11.450121 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-667f8ce9-1bcf-4c36-830e-fed3f514d0e0 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 15:10:11.451712 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-667f8ce9-1bcf-4c36-830e-fed3f514d0e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:10:11.521139 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1459/2912] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 15:10:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 15:10:11.549295 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdfb179-03cb-401f-8e52-d61da176ae55 admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:10:11.551442 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-2cdfb179-03cb-401f-8e52-d61da176ae55 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:10:11.609262 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdfb179-03cb-401f-8e52-d61da176ae55 admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:10:11.612884 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a0712e-d422-4576-983e-bdff2260883b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2790149639aa418bbc4e4982e93eda44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:10:11.613480 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a0712e-d422-4576-983e-bdff2260883b service neutron] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:10:11.613754 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a0712e-d422-4576-983e-bdff2260883b service neutron] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:10:11.614025 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a0712e-d422-4576-983e-bdff2260883b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:10:11.623609 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a0712e-d422-4576-983e-bdff2260883b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:10:11.625052 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a0712e-d422-4576-983e-bdff2260883b service neutron] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:10:11.630882 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 15:10:11.636895 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1454/2913] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 15:10:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 15:10:11.899126 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667f8ce9-1bcf-4c36-830e-fed3f514d0e0 None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:10:11.901186 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-667f8ce9-1bcf-4c36-830e-fed3f514d0e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:10:11.919910 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667f8ce9-1bcf-4c36-830e-fed3f514d0e0 None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:10:11.926570 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb14f42-dd37-4133-ab56-d1a6e33511dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:10:11.927400 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb14f42-dd37-4133-ab56-d1a6e33511dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:10:11.927784 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb14f42-dd37-4133-ab56-d1a6e33511dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:10:11.928156 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb14f42-dd37-4133-ab56-d1a6e33511dd admin admin] PATH_INFO: `/v3/projects/d429d17d00c24f22b74b56446fcaf35a` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:10:11.929989 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb14f42-dd37-4133-ab56-d1a6e33511dd admin admin] RBAC: Authorizing `identity:delete_project(project_id=d429d17d00c24f22b74b56446fcaf35a)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:10:11.932556 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb14f42-dd37-4133-ab56-d1a6e33511dd admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:10:11.997584 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-ccb14f42-dd37-4133-ab56-d1a6e33511dd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd429d17d00c24f22b74b56446fcaf35a', 'request_id': 'req-ccb14f42-dd37-4133-ab56-d1a6e33511dd'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:10:12.006515 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-ccb14f42-dd37-4133-ab56-d1a6e33511dd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd429d17d00c24f22b74b56446fcaf35a', 'request_id': 'req-ccb14f42-dd37-4133-ab56-d1a6e33511dd'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:10:12.021669 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-ccb14f42-dd37-4133-ab56-d1a6e33511dd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd429d17d00c24f22b74b56446fcaf35a', 'request_id': 'req-ccb14f42-dd37-4133-ab56-d1a6e33511dd'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:10:12.038549 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-ccb14f42-dd37-4133-ab56-d1a6e33511dd admin admin] The token cache is being invalidate because project d429d17d00c24f22b74b56446fcaf35a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81342) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 15:10:12.038918 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-ccb14f42-dd37-4133-ab56-d1a6e33511dd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:10:12.039664 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-ccb14f42-dd37-4133-ab56-d1a6e33511dd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:10:12.045412 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1460/2914] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Sep 1 15:10:11 2022] DELETE /identity/v3/projects/d429d17d00c24f22b74b56446fcaf35a => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 15:10:12.046866 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa3b9ac-0802-4dfc-8bd1-b89a3e4deb30 None None] REQUEST_METHOD: `POST` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:10:12.047152 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa3b9ac-0802-4dfc-8bd1-b89a3e4deb30 None None] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:10:12.047449 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa3b9ac-0802-4dfc-8bd1-b89a3e4deb30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:10:12.076938 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.auth.core [None req-5fa3b9ac-0802-4dfc-8bd1-b89a3e4deb30 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81341) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 15:10:12.078373 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-5fa3b9ac-0802-4dfc-8bd1-b89a3e4deb30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:10:12.162413 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1455/2915] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 15:10:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 15:10:12.166685 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28a4a72-80ee-4cba-9c2d-77a6fb8c167d None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:10:12.166980 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28a4a72-80ee-4cba-9c2d-77a6fb8c167d None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:10:12.167257 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28a4a72-80ee-4cba-9c2d-77a6fb8c167d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:10:12.192694 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-d28a4a72-80ee-4cba-9c2d-77a6fb8c167d None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 15:10:12.194515 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-d28a4a72-80ee-4cba-9c2d-77a6fb8c167d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:10:12.206355 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1461/2916] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 15:10:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 15:10:12.210783 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa3b9ac-0802-4dfc-8bd1-b89a3e4deb30 None None] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:10:12.225886 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa3b9ac-0802-4dfc-8bd1-b89a3e4deb30 None None] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:10:12.230966 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d383277d-b466-4805-b678-bce143f41a21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:10:12.231571 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d383277d-b466-4805-b678-bce143f41a21 admin admin] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:10:12.231851 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d383277d-b466-4805-b678-bce143f41a21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:10:12.232120 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d383277d-b466-4805-b678-bce143f41a21 admin admin] PATH_INFO: `/v3/domains` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:10:12.233404 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d383277d-b466-4805-b678-bce143f41a21 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81341) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 15:10:12.233722 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d383277d-b466-4805-b678-bce143f41a21 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:10:12.236414 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d383277d-b466-4805-b678-bce143f41a21 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:10:12.249980 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1456/2917] 10.222.0.33 () {58 vars in 1153 bytes} [Thu Sep 1 15:10:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 15:10:12.254400 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28a4a72-80ee-4cba-9c2d-77a6fb8c167d None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:10:12.267755 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28a4a72-80ee-4cba-9c2d-77a6fb8c167d None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:10:12.274316 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a657cb5b-0e27-45ef-869a-f8f47697abb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:10:12.275140 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a657cb5b-0e27-45ef-869a-f8f47697abb1 admin admin] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:10:12.275427 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a657cb5b-0e27-45ef-869a-f8f47697abb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:10:12.275798 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a657cb5b-0e27-45ef-869a-f8f47697abb1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:10:12.276547 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a657cb5b-0e27-45ef-869a-f8f47697abb1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:10:12.279694 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a657cb5b-0e27-45ef-869a-f8f47697abb1 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:10:12.304175 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d383277d-b466-4805-b678-bce143f41a21 admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:10:12.309891 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-d383277d-b466-4805-b678-bce143f41a21 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:10:12.319229 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1462/2918] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Sep 1 15:10:12 2022] POST /identity/v3/projects => generated 387 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 15:10:12.324108 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a657cb5b-0e27-45ef-869a-f8f47697abb1 admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:10:12.325058 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d383277d-b466-4805-b678-bce143f41a21 admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:10:12.387428 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bc77b1-f2a5-48cd-84ab-b25f471b195b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:10:12.388093 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bc77b1-f2a5-48cd-84ab-b25f471b195b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:10:12.388334 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bc77b1-f2a5-48cd-84ab-b25f471b195b admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:10:12.388628 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bc77b1-f2a5-48cd-84ab-b25f471b195b admin admin] PATH_INFO: `/v3/users/afa6212f31034e27a8d32f114937190e` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:10:12.389939 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bc77b1-f2a5-48cd-84ab-b25f471b195b admin admin] RBAC: Authorizing `identity:delete_user(user_id=afa6212f31034e27a8d32f114937190e)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:10:12.390138 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 15:10:12.391783 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bc77b1-f2a5-48cd-84ab-b25f471b195b admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:10:12.396539 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a657cb5b-0e27-45ef-869a-f8f47697abb1 admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:10:12.402718 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f030e435-7732-46cb-9e70-719ba1a6bf74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:10:12.403333 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f030e435-7732-46cb-9e70-719ba1a6bf74 admin admin] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:10:12.403613 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f030e435-7732-46cb-9e70-719ba1a6bf74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:10:12.403893 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f030e435-7732-46cb-9e70-719ba1a6bf74 admin admin] PATH_INFO: `/v3/users` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:10:12.404520 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f030e435-7732-46cb-9e70-719ba1a6bf74 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:10:12.406576 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f030e435-7732-46cb-9e70-719ba1a6bf74 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:10:12.429040 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1463/2919] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Sep 1 15:10:12 2022] POST /identity/v3/users => generated 294 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 15:10:12.433958 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f030e435-7732-46cb-9e70-719ba1a6bf74 admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:10:12.443554 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f030e435-7732-46cb-9e70-719ba1a6bf74 admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:10:12.448833 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0831dabb-2c37-40a1-8b58-29edca02a983 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:10:12.449421 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0831dabb-2c37-40a1-8b58-29edca02a983 admin admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:10:12.449659 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-39bc77b1-f2a5-48cd-84ab-b25f471b195b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'afa6212f31034e27a8d32f114937190e'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:10:12.449804 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0831dabb-2c37-40a1-8b58-29edca02a983 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:10:12.449949 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0831dabb-2c37-40a1-8b58-29edca02a983 admin admin] PATH_INFO: `/v3/roles` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:10:12.450641 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0831dabb-2c37-40a1-8b58-29edca02a983 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:10:12.452221 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0831dabb-2c37-40a1-8b58-29edca02a983 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:10:12.457193 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-39bc77b1-f2a5-48cd-84ab-b25f471b195b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'afa6212f31034e27a8d32f114937190e'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:10:12.457984 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-39bc77b1-f2a5-48cd-84ab-b25f471b195b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'afa6212f31034e27a8d32f114937190e'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:10:12.462166 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1464/2920] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 1 15:10:12 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 15:10:12.465954 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0831dabb-2c37-40a1-8b58-29edca02a983 admin admin] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:10:12.473137 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-39bc77b1-f2a5-48cd-84ab-b25f471b195b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'afa6212f31034e27a8d32f114937190e'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:10:12.473820 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-39bc77b1-f2a5-48cd-84ab-b25f471b195b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'afa6212f31034e27a8d32f114937190e'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:10:12.475933 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0831dabb-2c37-40a1-8b58-29edca02a983 admin admin] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:10:12.480465 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94eeee66-b358-4d77-b2aa-fb8dd7f2621e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:10:12.481003 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94eeee66-b358-4d77-b2aa-fb8dd7f2621e admin admin] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:10:12.481281 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94eeee66-b358-4d77-b2aa-fb8dd7f2621e admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:10:12.481548 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94eeee66-b358-4d77-b2aa-fb8dd7f2621e admin admin] PATH_INFO: `/v3/roles` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:10:12.482166 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94eeee66-b358-4d77-b2aa-fb8dd7f2621e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:10:12.483831 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94eeee66-b358-4d77-b2aa-fb8dd7f2621e admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:10:12.486699 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1457/2921] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Sep 1 15:10:12 2022] DELETE /identity/v3/users/afa6212f31034e27a8d32f114937190e => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 15:10:12.498632 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1465/2922] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 1 15:10:12 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 15:10:12.499797 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bc77b1-f2a5-48cd-84ab-b25f471b195b admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:10:12.501236 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-39bc77b1-f2a5-48cd-84ab-b25f471b195b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:10:12.512043 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 15:10:12.570487 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bc77b1-f2a5-48cd-84ab-b25f471b195b admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:10:12.575828 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d782aadc-999a-452d-8b99-3df0f5d41ade admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:10:12.576486 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d782aadc-999a-452d-8b99-3df0f5d41ade admin admin] REQUEST_METHOD: `PUT` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:10:12.576767 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d782aadc-999a-452d-8b99-3df0f5d41ade admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:10:12.577042 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d782aadc-999a-452d-8b99-3df0f5d41ade admin admin] PATH_INFO: `/v3/projects/4a0cea4629614845a8060bd58fb09892/users/733a8e8f3386444bb649b5b9a3ab2942/roles/5d1feef0e13d4874928bd86258c4e3b2` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:10:12.591547 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d782aadc-999a-452d-8b99-3df0f5d41ade admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a0cea4629614845a8060bd58fb09892, user_id=733a8e8f3386444bb649b5b9a3ab2942, role_id=5d1feef0e13d4874928bd86258c4e3b2)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:10:12.594217 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d782aadc-999a-452d-8b99-3df0f5d41ade admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:10:12.607884 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1458/2923] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 1 15:10:12 2022] PUT /identity/v3/projects/4a0cea4629614845a8060bd58fb09892/users/733a8e8f3386444bb649b5b9a3ab2942/roles/5d1feef0e13d4874928bd86258c4e3b2 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 15:10:12.617942 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3082b5b8-0d95-4fef-bdcb-21b66801ba5b None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:10:12.618808 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3082b5b8-0d95-4fef-bdcb-21b66801ba5b None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:10:12.619131 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3082b5b8-0d95-4fef-bdcb-21b66801ba5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:10:12.649690 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-3082b5b8-0d95-4fef-bdcb-21b66801ba5b None None] MFA Rules not processed for user `733a8e8f3386444bb649b5b9a3ab2942`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 15:10:12.651737 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-3082b5b8-0d95-4fef-bdcb-21b66801ba5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:10:12.664026 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d782aadc-999a-452d-8b99-3df0f5d41ade admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:10:12.666705 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-d782aadc-999a-452d-8b99-3df0f5d41ade admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:10:12.751347 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d782aadc-999a-452d-8b99-3df0f5d41ade admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:10:12.756122 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da026a96-f381-429c-938d-5a17cac7ebf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:10:12.756863 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da026a96-f381-429c-938d-5a17cac7ebf3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:10:12.757297 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da026a96-f381-429c-938d-5a17cac7ebf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:10:12.757530 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da026a96-f381-429c-938d-5a17cac7ebf3 admin admin] PATH_INFO: `/v3/projects/3e49151735934f21a2ea909842e36a05` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:10:12.758814 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da026a96-f381-429c-938d-5a17cac7ebf3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3e49151735934f21a2ea909842e36a05)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:10:12.761043 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da026a96-f381-429c-938d-5a17cac7ebf3 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:10:12.762214 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1466/2924] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 15:10:12 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 15:10:12.770932 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3082b5b8-0d95-4fef-bdcb-21b66801ba5b None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:10:12.772953 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-3082b5b8-0d95-4fef-bdcb-21b66801ba5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:10:12.818688 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-da026a96-f381-429c-938d-5a17cac7ebf3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3e49151735934f21a2ea909842e36a05', 'request_id': 'req-da026a96-f381-429c-938d-5a17cac7ebf3'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:10:12.825132 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-da026a96-f381-429c-938d-5a17cac7ebf3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3e49151735934f21a2ea909842e36a05', 'request_id': 'req-da026a96-f381-429c-938d-5a17cac7ebf3'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:10:12.833230 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-da026a96-f381-429c-938d-5a17cac7ebf3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3e49151735934f21a2ea909842e36a05', 'request_id': 'req-da026a96-f381-429c-938d-5a17cac7ebf3'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:10:12.834782 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3082b5b8-0d95-4fef-bdcb-21b66801ba5b None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:10:12.844353 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-da026a96-f381-429c-938d-5a17cac7ebf3 admin admin] The token cache is being invalidate because project 3e49151735934f21a2ea909842e36a05 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81341) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 15:10:12.844641 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-da026a96-f381-429c-938d-5a17cac7ebf3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:10:12.845288 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-da026a96-f381-429c-938d-5a17cac7ebf3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:10:12.846866 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1459/2925] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Sep 1 15:10:12 2022] DELETE /identity/v3/projects/3e49151735934f21a2ea909842e36a05 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 15:10:12.850778 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da026a96-f381-429c-938d-5a17cac7ebf3 admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:10:12.853238 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-da026a96-f381-429c-938d-5a17cac7ebf3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:10:12.876156 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 15:10:12.885445 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4bbe4e-546d-4b57-850e-06f49e6cec37 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3b81739e34e4e648f88b6d73fcb5e46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:10:12.886265 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4bbe4e-546d-4b57-850e-06f49e6cec37 service cinder] REQUEST_METHOD: `GET` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:10:12.886556 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4bbe4e-546d-4b57-850e-06f49e6cec37 service cinder] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:10:12.886833 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4bbe4e-546d-4b57-850e-06f49e6cec37 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:10:12.888596 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-4d4bbe4e-546d-4b57-850e-06f49e6cec37 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:10:12.910054 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 15:10:12.921169 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da026a96-f381-429c-938d-5a17cac7ebf3 admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:10:12.925964 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70dacc8f-337c-43fd-b55e-82922eef3fa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:10:12.926617 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dacc8f-337c-43fd-b55e-82922eef3fa2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:10:12.926895 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dacc8f-337c-43fd-b55e-82922eef3fa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:10:12.927167 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dacc8f-337c-43fd-b55e-82922eef3fa2 admin admin] PATH_INFO: `/v3/users/774adf3c3da84a78979a826f6f884922` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:10:12.928322 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70dacc8f-337c-43fd-b55e-82922eef3fa2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=774adf3c3da84a78979a826f6f884922)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:10:12.930257 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70dacc8f-337c-43fd-b55e-82922eef3fa2 admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:10:12.951218 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4bbe4e-546d-4b57-850e-06f49e6cec37 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:10:12.952651 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4bbe4e-546d-4b57-850e-06f49e6cec37 service cinder] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:10:12.962918 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1467/2926] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 15:10:12 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 15:10:12.978124 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-70dacc8f-337c-43fd-b55e-82922eef3fa2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '774adf3c3da84a78979a826f6f884922'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:10:12.985467 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-70dacc8f-337c-43fd-b55e-82922eef3fa2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '774adf3c3da84a78979a826f6f884922'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:10:12.986338 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-70dacc8f-337c-43fd-b55e-82922eef3fa2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '774adf3c3da84a78979a826f6f884922'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:10:12.998107 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-70dacc8f-337c-43fd-b55e-82922eef3fa2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '774adf3c3da84a78979a826f6f884922'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:10:12.998901 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-70dacc8f-337c-43fd-b55e-82922eef3fa2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '774adf3c3da84a78979a826f6f884922'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:10:13.011025 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1460/2927] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Sep 1 15:10:12 2022] DELETE /identity/v3/users/774adf3c3da84a78979a826f6f884922 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 15:10:13.194986 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4bbe4e-546d-4b57-850e-06f49e6cec37 service cinder] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:10:13.209196 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4bbe4e-546d-4b57-850e-06f49e6cec37 service cinder] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:10:13.284779 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e971b902-8247-4ef1-98a2-e5b2281c86f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:10:13.285410 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e971b902-8247-4ef1-98a2-e5b2281c86f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:10:13.285685 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e971b902-8247-4ef1-98a2-e5b2281c86f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:10:13.285953 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e971b902-8247-4ef1-98a2-e5b2281c86f9 admin admin] PATH_INFO: `/v3/projects/32962f5671bb4bbfa75d1d8efb7f684a` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:10:13.287279 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e971b902-8247-4ef1-98a2-e5b2281c86f9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=32962f5671bb4bbfa75d1d8efb7f684a)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:10:13.289412 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e971b902-8247-4ef1-98a2-e5b2281c86f9 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:10:13.340379 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-e971b902-8247-4ef1-98a2-e5b2281c86f9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '32962f5671bb4bbfa75d1d8efb7f684a', 'request_id': 'req-e971b902-8247-4ef1-98a2-e5b2281c86f9'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:10:13.346623 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-e971b902-8247-4ef1-98a2-e5b2281c86f9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '32962f5671bb4bbfa75d1d8efb7f684a', 'request_id': 'req-e971b902-8247-4ef1-98a2-e5b2281c86f9'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:10:13.355730 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-e971b902-8247-4ef1-98a2-e5b2281c86f9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '32962f5671bb4bbfa75d1d8efb7f684a', 'request_id': 'req-e971b902-8247-4ef1-98a2-e5b2281c86f9'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:10:13.367269 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-e971b902-8247-4ef1-98a2-e5b2281c86f9 admin admin] The token cache is being invalidate because project 32962f5671bb4bbfa75d1d8efb7f684a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81342) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 15:10:13.367551 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-e971b902-8247-4ef1-98a2-e5b2281c86f9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:10:13.368195 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-e971b902-8247-4ef1-98a2-e5b2281c86f9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:10:13.369652 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1468/2928] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Sep 1 15:10:13 2022] DELETE /identity/v3/projects/32962f5671bb4bbfa75d1d8efb7f684a => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 15:10:23.007348 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=44,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 15:10:23.364255 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=68,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 15:10:29.489831 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70dacc8f-337c-43fd-b55e-82922eef3fa2 admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:10:29.491251 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-70dacc8f-337c-43fd-b55e-82922eef3fa2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:10:29.509271 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70dacc8f-337c-43fd-b55e-82922eef3fa2 admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:10:29.564852 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0801031d-80c5-416f-a3b0-20886d5beeea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:10:29.565477 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0801031d-80c5-416f-a3b0-20886d5beeea admin admin] REQUEST_METHOD: `DELETE` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:10:29.565754 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0801031d-80c5-416f-a3b0-20886d5beeea admin admin] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:10:29.566031 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0801031d-80c5-416f-a3b0-20886d5beeea admin admin] PATH_INFO: `/v3/users/733a8e8f3386444bb649b5b9a3ab2942` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:10:29.567173 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0801031d-80c5-416f-a3b0-20886d5beeea admin admin] RBAC: Authorizing `identity:delete_user(user_id=733a8e8f3386444bb649b5b9a3ab2942)` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:10:29.569085 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0801031d-80c5-416f-a3b0-20886d5beeea admin admin] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:10:29.627279 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-0801031d-80c5-416f-a3b0-20886d5beeea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '733a8e8f3386444bb649b5b9a3ab2942'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:10:29.635412 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-0801031d-80c5-416f-a3b0-20886d5beeea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '733a8e8f3386444bb649b5b9a3ab2942'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:10:29.636247 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-0801031d-80c5-416f-a3b0-20886d5beeea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '733a8e8f3386444bb649b5b9a3ab2942'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:10:29.648380 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-0801031d-80c5-416f-a3b0-20886d5beeea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '733a8e8f3386444bb649b5b9a3ab2942'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:10:29.649076 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.notifications [None req-0801031d-80c5-416f-a3b0-20886d5beeea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '733a8e8f3386444bb649b5b9a3ab2942'} {{(pid=81341) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:10:29.666843 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1461/2929] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Sep 1 15:10:29 2022] DELETE /identity/v3/users/733a8e8f3386444bb649b5b9a3ab2942 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 15:10:29.671961 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e63b7f-9b7f-4ecf-ba55-bafbd19416b3 None None] REQUEST_METHOD: `POST` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:10:29.672393 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e63b7f-9b7f-4ecf-ba55-bafbd19416b3 None None] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:10:29.672506 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e63b7f-9b7f-4ecf-ba55-bafbd19416b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:10:29.692496 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.auth.core [None req-81e63b7f-9b7f-4ecf-ba55-bafbd19416b3 None None] MFA Rules not processed for user `abbabc07754b486599ec507190d24c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 15:10:29.693731 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-81e63b7f-9b7f-4ecf-ba55-bafbd19416b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:10:29.757120 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1469/2930] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 1 15:10:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 15:10:29.766799 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0801031d-80c5-416f-a3b0-20886d5beeea admin admin] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:10:29.768240 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-0801031d-80c5-416f-a3b0-20886d5beeea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:10:29.825736 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0801031d-80c5-416f-a3b0-20886d5beeea admin admin] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:10:29.829412 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae6d6ba-c4a4-44f0-827c-6b71ed8033c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2790149639aa418bbc4e4982e93eda44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:10:29.830008 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae6d6ba-c4a4-44f0-827c-6b71ed8033c7 service neutron] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:10:29.830317 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae6d6ba-c4a4-44f0-827c-6b71ed8033c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:10:29.830593 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae6d6ba-c4a4-44f0-827c-6b71ed8033c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:10:29.840024 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae6d6ba-c4a4-44f0-827c-6b71ed8033c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:10:29.841494 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae6d6ba-c4a4-44f0-827c-6b71ed8033c7 service neutron] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:10:29.850888 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1462/2931] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 15:10:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 15:10:30.112345 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e63b7f-9b7f-4ecf-ba55-bafbd19416b3 None None] Authenticating user token {{(pid=81342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:10:30.113884 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.fernet_utils [None req-81e63b7f-9b7f-4ecf-ba55-bafbd19416b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:10:30.125901 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e63b7f-9b7f-4ecf-ba55-bafbd19416b3 None None] Validating token access rules against request {{(pid=81342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:10:30.130462 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3392e36d-25c3-44a8-8ae9-59e1997b9310 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abbabc07754b486599ec507190d24c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cec9ba4a51d40d1be462a47013baf0a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:10:30.131085 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3392e36d-25c3-44a8-8ae9-59e1997b9310 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:10:30.131360 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3392e36d-25c3-44a8-8ae9-59e1997b9310 admin admin] SCRIPT_NAME: `/identity` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:10:30.131628 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3392e36d-25c3-44a8-8ae9-59e1997b9310 admin admin] PATH_INFO: `/v3/projects/4a0cea4629614845a8060bd58fb09892` {{(pid=81342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:10:30.132768 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3392e36d-25c3-44a8-8ae9-59e1997b9310 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4a0cea4629614845a8060bd58fb09892)` {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:10:30.134749 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3392e36d-25c3-44a8-8ae9-59e1997b9310 admin admin] RBAC: Authorization granted {{(pid=81342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:10:30.181705 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-3392e36d-25c3-44a8-8ae9-59e1997b9310 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4a0cea4629614845a8060bd58fb09892', 'request_id': 'req-3392e36d-25c3-44a8-8ae9-59e1997b9310'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:10:30.187861 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-3392e36d-25c3-44a8-8ae9-59e1997b9310 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4a0cea4629614845a8060bd58fb09892', 'request_id': 'req-3392e36d-25c3-44a8-8ae9-59e1997b9310'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:10:30.194724 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-3392e36d-25c3-44a8-8ae9-59e1997b9310 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4a0cea4629614845a8060bd58fb09892', 'request_id': 'req-3392e36d-25c3-44a8-8ae9-59e1997b9310'} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:10:30.205251 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-3392e36d-25c3-44a8-8ae9-59e1997b9310 admin admin] The token cache is being invalidate because project 4a0cea4629614845a8060bd58fb09892 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81342) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 15:10:30.205533 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-3392e36d-25c3-44a8-8ae9-59e1997b9310 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:10:30.206171 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG keystone.notifications [None req-3392e36d-25c3-44a8-8ae9-59e1997b9310 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 15:10:30.207664 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: [pid: 81342|app: 0|req: 1470/2932] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Sep 1 15:10:30 2022] DELETE /identity/v3/projects/4a0cea4629614845a8060bd58fb09892 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 15:10:39.846813 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 15:10:40.202353 dv-cinder-855058-2-iscsi devstack@keystone.service[81342]: DEBUG dbcounter [-] [81342] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 15:11:43.111504 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae6d6ba-c4a4-44f0-827c-6b71ed8033c7 service neutron] Authenticating user token {{(pid=81341) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 15:11:43.112918 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-cae6d6ba-c4a4-44f0-827c-6b71ed8033c7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:11:43.179944 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae6d6ba-c4a4-44f0-827c-6b71ed8033c7 service neutron] Validating token access rules against request {{(pid=81341) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 15:11:43.237475 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5381bc86-e833-438f-82aa-2e1d54f1502d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f312c8dcabe48ef9ea453364ade38ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8c85b4bd17e4e89b56bf0f831b474d5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81341) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 15:11:43.238074 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5381bc86-e833-438f-82aa-2e1d54f1502d service placement] REQUEST_METHOD: `GET` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 15:11:43.238383 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5381bc86-e833-438f-82aa-2e1d54f1502d service placement] SCRIPT_NAME: `/identity` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 15:11:43.238655 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5381bc86-e833-438f-82aa-2e1d54f1502d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81341) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 15:11:43.240389 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.fernet_utils [None req-5381bc86-e833-438f-82aa-2e1d54f1502d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81341) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 15:11:43.251827 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5381bc86-e833-438f-82aa-2e1d54f1502d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 15:11:43.253411 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5381bc86-e833-438f-82aa-2e1d54f1502d service placement] RBAC: Authorization granted {{(pid=81341) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 15:11:43.263123 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: [pid: 81341|app: 0|req: 1463/2933] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 1 15:11:43 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 15:11:53.258992 dv-cinder-855058-2-iscsi devstack@keystone.service[81341]: DEBUG dbcounter [-] [81341] Writing DB stats keystone:SELECT=58 {{(pid=81341) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}